Testing Events

Testing events covers the subjects and predicates related to test-execution performed either independently or as part of CI/CD pipelines.

These subjects replace the previously defined testCase/testSuite subjects in the Continuous Integration category

Subjects

This specification defines three subjects in this stage: testCaseRun, testSuiteRun and testOutput.

The predicates for testCaseRun must follow the following expectations:

• The following events are valid initial events: queued, started, or skipped.
• The following events are valid terminal events that will not be followed by other events: finished and skipped.
• A queued event must be followed by one of the following: started or finished.
• A started event must be followed by a finished event.

Subject	Description	Predicates
testCaseRun	The execution of a software testCase	queued, started, finished, skipped
testSuiteRun	The execution of a software testSuite	queued, started, finished
testOutput	An output artifact produced by a testCaseRun	published

testCaseRun

A testCaseRun is a process that executes a test against an input software artifact of some kind, for instance source code, a binary, a container image or else. A testCaseRun is the smallest unit of testing that the user wants to track, testSuiteRuns are for grouping purposes.

Field	Type	Description	Examples
id	String	Uniquely identifies the subject within the source.	Login-Test-execution-1213423, e2e-test1, 2334234324
source	URI-Reference	source from the context	staging/testkube, testkube-dev-123
environment	Object environment	The environment in which this testCaseRun is executing	{"id": "1234"}, {"id": "dev", "source": "testkube-dev-123"}
testCase	Object testCase	An optional definition of the testCase being executed	{"id": "92834723894", "name": "Login Test", "type": "integration"}
testSuiteRun	Object testSuiteRun	An optional testSuiteRun to associate this testCaseRun with a containing testSuiteRun	{"id":"Auth-TestSuite-execution-12334", "source": "staging/testkube"}

testSuiteRun

A testSuiteRun represents the execution of a set of one or more testCaseRuns.

Field	Type	Description	Examples
id	String	Uniquely identifies the subject within the source.	Auth-TestSuite-execution-12334, regression-123
source	URI-Reference	source from the context	staging/testkube, testkube-dev-123
environment	Object environment	The environment in which this testSuiteRun is executing	{"id": "1234"}, {"id": "dev", "source": "testkube-dev-123"}
testSuite	Object testSuite	An optional definition of the testSuite being executed	{"id": "92834723894", "name": "Auth TestSuite"}

testOutput

One or more testOutput artifacts are usually produced as the result of a test execution.

Field	Type	Description	Examples
id	String	Uniquely identifies the subject within the source.	report-23123123
source	URI-Reference	source from the context	staging/testkube, testkube-dev-123
outputType	String (enum)	The type of output, one of report, video, image, log, other	video
format	String	The Content-Type of the output artifact	application/pdf, image/png, application/json
uri	URI-Reference	A reference to retrieve the specified output artifact	https://testkube.mycluster.internal/artifacts/23123123
testCaseRun	Object testCaseRun	An optional testCaseRun to link this artifact to a specific testCaseRun	{"id":"Login-Test-execution-1213423", "source": "staging/testkube"}

Events

testCaseRun queued

This event represents when a testCaseRun has been queued for execution - and is waiting for applicable preconditions (resource availability, other tasks, etc.) to be fulfilled before actually executing.

• Event Type: dev.cdevents.testcaserun.queued.0.3.0
• Predicate: queued
• Subject: testCaseRun

Field	Type	Description	Examples	Required
id	String	Uniquely identifies the subject within the source.	unitest-abc, e2e-test1, scan-image1	✅
source	URI-Reference	source from the context
environment	Object environment	The environment in which this testCaseRun is queued	{"id": "1234"}, {"id": "dev", "source": "testkube-dev-123"}	✅
testCase	Object testCase	Definition of the testCase being executed	{"id": "92834723894", "name": "Login Test", "type": "integration"}
testSuiteRun	Object testSuiteRun	A testSuiteRun to associate this testCaseRun with a containing testSuiteRun	{"id":"Auth-TestSuite-execution-12334", "source": "staging/testkube"}
trigger	Object trigger	What triggered this testSuiteRun	{"type": "schedule"}

testCaseRun started

This event represents a started testCase execution.

• Event Type: dev.cdevents.testcaserun.started.0.3.0
• Predicate: started
• Subject: testCaseRun

Field	Type	Description	Examples	Required
id	String	Uniquely identifies the subject within the source.	unitest-abc, e2e-test1, scan-image1	✅
source	URI-Reference	source from the context
environment	Object environment	The environment in which this testCaseRun is running	{"id": "1234"}, {"id": "dev", "source": "testkube-dev-123"}	✅
testCase	Object testCase	Definition of the testCase being executed	{"id": "92834723894", "name": "Login Test", "type": "integration"}
testSuiteRun	Object testSuiteRun	A testSuiteRun to associate this testCaseRun with a containing testSuiteRun	{"id":"Auth-TestSuite-execution-12334", "source": "staging/testkube"}
trigger	Object trigger	What triggered this testSuiteRun	{"type": "event"}

testCaseRun finished

This event represents a finished testCase execution. The event will contain the outcome and additional metadata as applicable.

• Event Type: dev.cdevents.testcaserun.finished.0.3.0
• Predicate: finished
• Subject: testCaseRun

Field	Type	Description	Examples	Required
id	String	Uniquely identifies the subject within the source.	unitest-abc, e2e-test1, scan-image1	✅
source	URI-Reference	source from the context
environment	Object environment	The environment in which this testCaseRun was running	{"id": "1234"}, {"id": "dev", "source": "testkube-dev-123"}	✅
testCase	Object testCase	Definition of the testCase being executed	{"id": "92834723894", "name": "Login Test", "type": "integration"}
testSuiteRun	Object testSuiteRun	A testSuiteRun to associate this testCaseRun with a containing testSuiteRun	{"id":"Auth-TestSuite-execution-12334", "source": "staging/testkube"}
outcome	String (enum)	The outcome of the testSuite execution, one of success, failure, cancel, or error	success, failure, cancel, error	✅
severity	String (enum)	Severity if the test failed, one of low, medium, high, critical	critical
reason	String	A reason related to the outcome of the execution	Canceled by user, Failed assertion, Timed out

testCaseRun skipped

This event represents a skipped testCaseRun execution. The event should only be emitted if there has been no prior "queued" or "started" event.

• Event Type: dev.cdevents.testcaserun.skipped.0.2.0
• Predicate: skipped
• Subject: testCaseRun

Field	Type	Description	Examples	Required
id	String	Uniquely identifies the subject within the source.	unitest-abc, e2e-test1, scan-image1	✅
source	URI-Reference	source from the context
environment	Object environment	The environment in which this testCaseRun would have run, but was skipped.	{"id": "1234"}, {"id": "dev", "source": "testkube-dev-123"}	✅
testCase	Object testCase	Definition of the testCase being executed	{"id": "92834723894", "name": "Login Test", "type": "integration"}
testSuiteRun	Object testSuiteRun	A testSuiteRun to associate this testCaseRun with a containing testSuiteRun	{"id":"Auth-TestSuite-execution-12334", "source": "staging/testkube"}
severity	String	Severity if the test failed, one of low, medium, high, critical	critical
reason	String	A reason for skipping the test case run.	Not running in given environment, Skipping slow tests

testSuiteRun queued

This event represents when a testSuiteRun has been queued for execution - and is waiting for applicable preconditions (resource availability, other tasks, etc.) to be met before actually executing.

• Event Type: dev.cdevents.testsuiterun.queued.0.3.0
• Predicate: queued
• Subject: testSuiteRun

Field	Type	Description	Examples	Required
id	String	Uniquely identifies the subject within the source.	unitest-abc, e2e-test1, scan-image1	✅
source	URI-Reference	source from the context
environment	Object environment	The environment in which this testSuiteRun is queued	{"id": "1234"}, {"id": "dev", "source": "testkube-dev-123"}	✅
testSuite	Object testSuite	Definition of the testSuite being executed	{"id": "92834723894", "name": "Auth TestSuite"}
trigger	Object trigger	What triggered this testSuiteRun	{"type": "manual"}

testSuiteRun started

This event represents a started testSuite execution.

• Event Type: dev.cdevents.testsuiterun.started.0.3.0
• Predicate: started
• Subject: testSuiteRun

Field	Type	Description	Examples	Required
id	String	Uniquely identifies the subject within the source.	unit, e2e, security	✅
source	URI-Reference	source from the context
environment	Object environment	The environment in which this testSuiteRun is running	{"id": "1234"}, {"id": "dev", "source": "testkube-dev-123"}	✅
testSuite	Object testSuite	Definition of the testSuite being executed	{"id": "92834723894", "name": "Auth TestSuite"}
trigger	Object trigger	What triggered this testSuiteRun	{"type": "pipeline"}

testSuiteRun finished

This event represents a finished testSuite execution. The event will contain the outcome and additional metadata as applicable.

• Event Type: dev.cdevents.testsuiterun.finished.0.3.0
• Predicate: finished
• Subject: testSuiteRun

Field	Type	Description	Examples	Required
id	String	Uniquely identifies the subject within the source.	unit, e2e, security	✅
source	URI-Reference	source from the context
environment	Object environment	The environment in which this testSuiteRun was running	{"id": "1234"}, {"id": "dev", "source": "testkube-dev-123"}	✅
outcome	String (enum)	The outcome of the testSuite execution, one of success, failure, cancel, or error	success, failure, cancel, error	✅
severity	String (enum)	Severity if the test failed, one of low, medium, high, critical	critical, low, medium, high
reason	String	A reason related to the outcome of the execution	Canceled by user, Failed testCase
testSuite	Object testSuite	Definition of the testSuite being executed	{"id": "92834723894", "name": "Auth TestSuite"}

testOutput published

The event represents a test execution output that has been published.

• Event Type: dev.cdevents.testoutput.published.0.3.0
• Predicate: published
• Subject: testOutput

Field	Type	Description	Examples	Required
id	String	Uniquely identifies the subject within the source.	12312334	✅
source	URI-Reference	source from the context		✅
outputType	String (enum)	The type of output, one of report, video, image, log, other	video	✅
format	String	The Content-Type of the output artifact	application/pdf, image/png, application/json	✅
uri	URI-Reference	A reference to retrieve the specified output artifact	https://testkube.mycluster.internal/artifacts/23123123
testCaseRun	Object testCaseRun	An optional testCaseRun to link this artifact to a specific testCaseRun	{"id":"Login-Test-execution-1213423", "source": "staging/testkube"}

Common Objects

trigger

A trigger in this context is what started a corresponding testCaseRun/testSuiteRun.

Field	Type	Description	Examples	Required
type	String (enum)	The type of trigger, one of manual, pipeline, event, schedule, other		✅
uri	URI-Reference	A uri reference to this trigger

testCase

A testCase is the actual test that is being run by a testCaseRun.

Field	Type	Description	Examples	Required
id	String	Uniquely identifies the testCase within a test management system.	12312334	✅
type	String (enum)	The type of test, one of performance, functional, unit, security, compliance, integration, e2e, other	functional
name	String	A user-friendly name for this testCase	Login Test
version	String	The version of the testCase	1.0
uri	URI-Reference	A uri reference to this testCase

testSuite

A testSuite is a collection of testCase objects managed in an external system. Each time a testSuite is executed the corresponding testSuiteXXX events should be emmitted.

Field	Type	Description	Examples	Required
id	String	Uniquely identifies the testSuite within a test management system.	12312334	✅
name	String	A user-friendly name for this testSuite	Auth TestSuite
version	String	The version of the testSuite	1.0
uri	URI-Reference	A uri reference to this testSuite