Contributing a TMGoat fixture

A fixture = one system at one difficulty, with inputs and a ground-truth answer key.

Steps

1. fixtures/<NN-sector>/<tier>/ — create the folder.
2. manifest.yaml — tag it; validate against schema/manifest.schema.json.
3. inputs/ — the artifacts a tool ingests: diagrams/ (drawio preferred), intake/ (docs + OpenAPI), code/, iac/. Keep component names consistent across artifacts — cross-source correlation is part of what we test.
4. solution/threat-model.yaml — the answer key; validate against schema/threat-model.schema.json. Mark every intentional threat planted: true and give it cwe + standards mappings.
5. python harness/score.py ... to sanity-check, then add a row to catalog.md.

Tagging guidance (the important rule)

Difficulty is not size. Tag the three axes honestly in manifest.yaml: architectural_complexity, threat_subtlety, input_completeness.

Deliberately break the size↔subtlety correlation. We want fixtures that are:

• small but subtle (few components, one nasty design flaw), and
• large but obvious (many components, only textbook issues).

If every "difficult" fixture is just "bigger," the benchmark only measures parsing, not reasoning.

Robustness fixtures

Most fixtures are input_completeness: rich (all artifacts present and agreeing). Please also contribute partial (e.g. code + a vague description, no diagram) and adversarial (contradictory artifacts) variants — that's where automated modelers actually break.

Safety

These designs contain intentional weaknesses. Label them as such; never present fixture code/IaC as production guidance.