SharpTS Test262 Runner
June 6, 2026 · View on GitHub
Runs SharpTS against the canonical TC39 Test262 ECMA-262 conformance suite in both interpreter and compiled-IL modes, diffing outcomes against committed baselines. The TS-conformance equivalent lives in SharpTS.TypeScriptConformance/.
Initial setup
git submodule update --init external/test262
Running locally
This project is not included in SharpTS.sln and won't be picked up by solution-level dotnet test. Invoke explicitly:
dotnet test SharpTS.Test262/SharpTS.Test262.csproj
The default subset (config/subset.json) keeps a runtime budget of a few minutes. The wide-sweep config (config/wide-sweep.json) exercises a much larger slice and writes a markdown report instead of diffing — useful for periodic deep checks.
Updating the baselines
SHARPTS_TEST262_UPDATE_BASELINE=1 dotnet test SharpTS.Test262/SharpTS.Test262.csproj
Writes baselines/interpreted.txt and baselines/compiled.txt. Commit the regenerated files alongside the change so reviewers can see what shifted.
SHARPTS_TEST262_WIDE_SWEEP=1 dotnet test SharpTS.Test262/SharpTS.Test262.csproj
Switches to the wide-sweep config and writes wide-sweep-report.md instead of diffing. Long-running.
Bucket model
| Bucket | Meaning |
|---|---|
Pass | Test body completed without an assertion throwing. |
Fail | Test body threw a Test262Error (assertion failed). |
ParseError | Source (or assembled harness) failed to lex/parse. |
TypeCheckError | Static type checker rejected the source. |
RuntimeError | Test body threw something other than Test262Error. |
Timeout | Execution exceeded the per-test deadline. |
HarnessError | Harness code (sta.js / assert.js / includes) threw before the test body ran. |
Skipped | Intentionally not run (negative test, deferred feature, skip-list match). |
Skip reasons are appended to the bucket (Skipped:async-done-deferred) so the diff harness can tell different skip causes apart.
Layout
| Path | Purpose |
|---|---|
external/test262/ | Vendored Test262 repo (submodule, shallow) |
config/subset.json | Default subset: folders to run, per-test timeout, skip-features file |
config/wide-sweep.json | Larger periodic-sweep config; writes a report instead of diffing |
config/skip-features.txt | Feature tags (generators, Atomics, decorators, ...) that cause a test to be skipped |
baselines/interpreted.txt | Committed baseline for interpreter mode |
baselines/compiled.txt | Committed baseline for compiled-IL mode |
See also
SharpTS.TypeScriptConformance/— equivalent for the TypeScript conformance corpus.