SEP-1913 Trust & Sensitivity Annotations

March 15, 2026 · View on GitHub

Research repository for SEP-1913 — adding trust and sensitivity annotations to the Model Context Protocol.

This repo contains the Python SDK implementation, usability study materials, demos, and documentation for the SEP-1913 proposal.

Project Structure

mcp-trust-annotations/
├── README.md                          # This file (repo overview)
│
├── packages/python/                   # Python SDK package
│   ├── README.md                      # SDK setup, usage & test instructions
│   ├── pyproject.toml                 # Build config (hatchling)
│   │
│   ├── src/mcp_trust/                 # Library source code
│   │   ├── __init__.py                # Public API exports
│   │   ├── types.py                   # Core types: DataClass, Destination, Source, etc.
│   │   ├── annotate.py                # @trust_annotated decorator, to_wire/from_wire
│   │   ├── emit.py                    # Structured audit logging (JSON)
│   │   ├── policy.py                  # PolicyEngine: audit/warn/enforce modes
│   │   └── propagate.py               # SessionTracker: sensitivity escalation
│   │
│   ├── tests/                         # Test suite (138 tests)
│   │   ├── test_types.py              # Core type construction & ordering (26 tests)
│   │   ├── test_annotate.py           # Decorator & wire-format roundtrip (27 tests)
│   │   ├── test_emit.py               # Audit logging (6 tests)
│   │   ├── test_policy.py             # Policy engine rules & modes (28 tests)
│   │   ├── test_propagate.py          # Session tracking & escalation (16 tests)
│   │   └── test_usability_scenarios.py # Usability study oracle tests (35 tests)
│   │
│
├── demo/                              # Demo applications
│   ├── server/mcp_server.py           # MCP server with annotated tools
│   ├── client/mcp_client.py           # MCP client consuming annotations
│   ├── host/host.py                   # Host orchestrator
│   ├── multi-agent/data_leak_prevention.py  # Multi-agent DLP scenario
│   └── dashboard/                     # Web dashboard
│       ├── app.py                     # Dashboard backend
│       ├── index.html                 # Dashboard UI
│       └── start_dashboard.ps1        # Launch script (Windows)
│
├── docs/                              # Documentation
│   ├── api-reference.md               # Full API reference
│   ├── engineering-architecture.md    # Architecture & design decisions
│   ├── diagrams.py                    # Diagram generation (Pillow/PIL)
│   └── generate_doc.py               # Word document generator (python-docx)
│
├── contrib/                           # Community contributions
│   └── usability-study/              # Study protocol, scripts & results
│       ├── protocol.md
│       ├── tool-corpus.json
│       ├── USABILITY_SCENARIOS.md     # Participant worksheet (Markdown)
│       ├── SEP1913_Usability_Scenarios.xlsx  # Participant worksheet (Excel)
│       ├── run_llm_study.py           # LLM-based usability study runner
│       ├── study_config.py            # LLM study configuration & oracle
│       ├── generate_usability_workbook.py   # Excel workbook generator
│       ├── study_cache/               # Cached LLM study results
│       └── results/
│
└── _diagrams/                         # Generated diagram images (fig1–fig14)

Getting Started

For full setup, installation, and usage instructions, see the Python SDK README.

Quick Setup

# 1. Clone and enter the repo
git clone https://github.com/YOUR_HANDLE/mcp-trust-annotations.git
cd mcp-trust-annotations

# 2. Create and activate a virtual environment
python -m venv .venv
.\.venv\Scripts\Activate.ps1        # Windows PowerShell
# source .venv/bin/activate          # macOS / Linux

# 3. Install the SDK with dev dependencies

pip install -e ".[dev]"

# 4. Run all tests (138 tests)
pytest tests/ -v

Demo

Run the healthcare MCP server demo:

Move to workspace root (mcp-trust-annotations)
cd demo/server
PYTHONPATH=../../packages/python/src python mcp_server.py

On Windows (PowerShell):

$env:PYTHONPATH = "packages\python\src"
python demo\server\mcp_server.py

Interactive Dashboard

The test dashboard provides a web-based UI for exploring the MCP server's tools, running scenarios, and observing SEP-1913 annotations in action.

Prerequisites

Install the dashboard dependencies (in addition to the SDK dev install):

pip install mcp starlette uvicorn

Starting the Dashboard

The easiest approach is the launch script (Windows PowerShell):

cd demo/dashboard
.\start_dashboard.ps1            # defaults to http://localhost:8913
.\start_dashboard.ps1 -Port 9000 # custom port

The script automatically activates the venv, checks for required packages, verifies the MCP server can start, frees the port if occupied, and launches the dashboard.

Alternatively, start manually:

$env:PYTHONPATH = "packages\python\src"
python demo\dashboard\app.py     # opens at http://localhost:8913

What the Dashboard Does

TabDescription
ToolsLists all 6 healthcare server tools with their full SEP-1913 annotations (sensitivity, source, destination, outcomes, hints)
Single Tool CallExecute any tool with custom arguments and inspect the returned result alongside its trust annotations
Multi-Agent DLPRun the data leak prevention scenario — demonstrates how SessionTracker escalation and PolicyEngine enforcement prevent sensitive data from leaking across agent boundaries
Policy TestsInteractively test policy rules (e.g., block credentials to public, warn on PII forward) and see PolicyDecision results
Session StateView real-time SessionTracker propagation state as tools are called in sequence

Tallying Responses Against the Usability Worksheet

After running scenarios in the dashboard, compare the observed behavior against the expected answers in the usability study:

  1. Open the worksheet — packages/python/USABILITY_SCENARIOS.md contains 10 scenarios that ask participants to predict annotation behavior (sensitivity levels, policy decisions, session escalation, etc.).

  2. Run matching scenarios in the dashboard — Each worksheet scenario maps to a tool or combination of tools in the MCP server. For example:

    • Scenario 1 (Email Send Tool) → call send_email and check destination / outcomes
    • Scenario 4 (Patient Lookup) → call patient_lookup and verify the regulated(HIPAA) sensitivity
    • Scenario 7 (Session Escalation) → call tools in sequence and watch the Session State tab escalate

  3. Score against the oracle — The oracle test suite at packages/python/tests/test_usability_scenarios.py encodes the correct answers for all 10 scenarios. Run it to validate:

    cd packages/python
PYTHONPATH=src pytest tests/test_usability_scenarios.py -v

  4. Record in the spreadsheet — contrib/usability-study/SEP1913_Usability_Scenarios.xlsx provides a template for tallying participant responses against the oracle answers. Mark each scenario as correct/incorrect and note any areas where the spec was ambiguous.

Usability Study

The SEP-1913 usability study evaluates whether developers (and LLMs) can correctly predict annotation behavior from the spec alone. It measures inter-rater agreement across 10 scenarios and identifies areas where the spec is clear, ambiguous, or has gaps.

Study Resources

ResourcePath
Participant worksheet (Markdown)contrib/usability-study/USABILITY_SCENARIOS.md
Participant worksheet (Excel)contrib/usability-study/SEP1913_Usability_Scenarios.xlsx
Oracle test suitepackages/python/tests/test_usability_scenarios.py
LLM study runnercontrib/usability-study/run_llm_study.py
Study protocolcontrib/usability-study/protocol.md
Tool corpuscontrib/usability-study/tool-corpus.json
Resultscontrib/usability-study/results/

How to Run the Study (Human Participants)

  1. Provide materials — Give the participant the SEP-1913 spec and the worksheet (USABILITY_SCENARIOS.md or the .xlsx version). Do not provide the SDK source code — the study tests spec clarity, not implementation knowledge.

  2. Participant completes the worksheet — 10 scenarios, each with 2–4 multiple-choice or short-answer questions about expected annotation values (Destination, Source, Outcome, sensitivity levels, policy decisions, session escalation). Estimated time: 30–45 minutes.

  3. Score against the oracle test suite — The oracle encodes the correct answers for all 10 scenarios:

    cd packages/python
PYTHONPATH=src pytest tests/test_usability_scenarios.py -v

    Each test name maps to a scenario + question (e.g., test_scenario_1_email_destination). Compare the participant's answers to the test expectations.

  4. Record results — Use the Excel worksheet at contrib/usability-study/SEP1913_Usability_Scenarios.xlsx to tally correct/incorrect per scenario per participant.

  5. (Optional) Validate with the dashboard — Start the interactive dashboard and run the matching scenarios live to let participants see actual behavior alongside their predictions.

How to Run the LLM Study

The LLM study sends the same 10 scenarios to multiple LLMs and auto-scores their responses against the oracle.

Prerequisites:

pip install anthropic openai google-generativeai openpyxl

Environment variables (set the keys for the models you want to test):

$env:ANTHROPIC_API_KEY = "sk-..."    # Claude Sonnet / Haiku
$env:OPENAI_API_KEY = "sk-..."       # GPT-4o / GPT-4o Mini
$env:GOOGLE_API_KEY = "..."          # Gemini 2.0 Flash

Run:

cd contrib/usability-study
PYTHONPATH=../../packages/python/src python run_llm_study.py

On Windows (PowerShell):

$env:PYTHONPATH = "packages\python\src"
python contrib\usability-study\run_llm_study.py

Results are cached in contrib/usability-study/study_cache/ and a summary Excel report is generated in contrib/usability-study/.

License

MIT

Important Context

  • The SDK maintains 1:1 schema fidelity with SEP-1913 — type names and wire format must match the spec exactly
  • DataClass is a union type: either a SimpleDataClass enum or a Regulated dataclass with regulatory scopes
  • TypeScript and C# SDKs are planned but not yet implemented
  • The project is pre-publication (not yet on PyPI) — awaiting SEP-1913 merge into MCP spec