contextweaver

Authority: This file is the single source of truth for agent-facing guidance. Tool-specific files (.claude/CLAUDE.md, .github/copilot-instructions.md) contain only overrides and guardrails — they defer to this file for all shared rules.

Purpose

contextweaver is a Python library for dynamic context management for tool-using AI agents. It provides two integrated engines:

1. Context Engine — phase-specific budgeted context compilation with a context firewall
2. Routing Engine — bounded-choice navigation over large tool catalogs via DAG + beam search

Non-goals: contextweaver is not an LLM inference layer and not a tool execution runtime. It prepares context and routes tools but never calls models or executes tools.

Module Map

Path	Responsibility
types.py	Core dataclasses and enums: SelectableItem, ContextItem, Phase, ItemKind, Sensitivity
envelope.py	Result types: ResultEnvelope, BuildStats, DroppedItem, ContextPack, ChoiceCard, HydrationResult, RoutingDecision
diagnostics.py	Versioned, payload-safe gateway events and sinks (DiagnosticEvent, DiagnosticSink, JSONL/in-memory sinks) plus deterministic aggregate reports (issues #370/#378).
inspection.py	Pure JSON/Markdown report construction for offline context, routing, and artifact inspection without raw payload content (issue #398).
telemetry_contract.py	Versioned JSONL telemetry handoff contract over DiagnosticEvent for downstream analytics (issue #382): EVENT_FAMILIES (8 families → event-name prefixes; shortlist/policy_denial/visibility reserved until their emitters land), classify_event, validate_event_dict (envelope shape + payload-leak heuristic), export_jsonl/read_jsonl (skip-and-collect). Hand-written envelope schema under schemas/telemetry/v1/; contract doc docs/telemetry.md.
ops_view.py	Read-only gateway ops-triage view over the diagnostics JSONL (issue #668): windowed OpsSnapshot (family counts, error rate, browse/execute latency percentiles, top executed/failing tools), plain-text + Rich renders, incremental file-tailing watch_loop. Backs mcp ops.
visualize.py	Self-contained HTML reports for RouteTrace / BuildStats / diagnostic-event timelines (issue #442): inline CSS only, all dynamic (untrusted) text HTML-escaped, byte-identical output for identical inputs. Backs the visualize CLI command.
config.py	Configuration: ContextBudget, ContextPolicy (incl. overflow_action budget-overflow policy, issue #510), and re-exported ScoringConfig
_scoring_config.py	ScoringConfig — candidate-scorer weights, incl. kind_priority + per-Phase phase_overrides (issue #487). Extracted from config.py to keep it ≤300 lines; re-exported there so from contextweaver.config import ScoringConfig is unchanged.
profiles.py	Routing and profile config: Mode, RoutingConfig, ProfileConfig, named presets
protocols.py	Protocol interfaces: TokenEstimator, EventHook, Summarizer, Extractor, RedactionHook, SensitivityClassifier (ingestion-time labelling, issue #542), MemorySource, Labeler, Retriever, Reranker, ClusteringEngine, RoutingScoreProvider (store protocols re-exported from store/protocols.py). Bundled estimators: HeuristicEstimator (default, script-aware, dependency-free — counts CJK/Kana/Hangul/emoji ≈1 token/char, issue #525), CharDivFourEstimator (raw len // 4 primitive), TiktokenEstimator (exact, falls back to HeuristicEstimator offline). Each carries a stable name for BuildStats.token_estimator.
store/protocols.py	Store-layer protocols: EventLog, ArtifactStore, EpisodicStore, FactStore
store/async_protocols.py	Async counterparts AsyncEventLog / AsyncArtifactStore / AsyncEpisodicStore / AsyncFactStore (issue #495) — same surface, async def. Consumed only by the async context/ path; backend-agnostic.
store/async_bridge.py	to_async(sync_store) — wraps a thread-safe sync backend as the matching async protocol via asyncio.to_thread. Thread-affine backends (SqliteEventLog, check_same_thread=True) are not valid targets (issue #495).
store/_async_to_sync.py	Inverse bridges + to_sync(async_store, loop) + is_async_store() (issue #495). Drives async stores on a private _LoopThread so the existing sync pipeline can consume them; ContextManager offloads build to a worker thread when async-backed. Not public API.
store/_loop_thread.py	The private _LoopThread (asyncio loop on a daemon thread) backing _async_to_sync (extracted to keep that module ≤300 lines). run(coro, timeout=…) bounds each op and raises StoreTimeoutError instead of hanging a stuck backend (issue #750). Not public API.
exceptions.py	Custom exception hierarchy (all errors inherit ContextWeaverError). Each class carries a stable, frozen code (e.g. CW_CONFIG) plus an optional hint; str(exc) renders [code] message (hint: …). Codes are documented in docs/errors.md and golden-listed in tests/test_exceptions.py (issues #635, #637).
_utils.py	Text similarity primitives: tokenize(), jaccard(), TfIdfScorer
secrets.py	Pure, deterministic secret detection/scrubbing primitives: scrub_secrets(), scrub_secrets_in_list(), contains_secret(), SecretPattern (issue #428). Shared by the firewall secret-scrub, the SecretRedactor hook, the sensitivity classifier, and ChoiceCard scrubbing. No I/O; never weakens a surface (only removes characters).
_version.py	Single-source version derived from importlib.metadata; fallback "0.0.0+local"
_incident_pack.py	Offline incident-pack builder for contextweaver mcp incident-pack (issue #661): creates bounded redacted zip bundles with a machine-readable manifest, summaries, redacted config/catalog/diagnostics excerpts, and reproduction checklist. Private; no public API.
_incident_pack_files.py	Private archive-entry helpers for incident packs: key-aware structured redaction, per-file truncation, hashes/timestamps for manifest entries, and best-effort JSON/YAML/JSONL parsing. Not public API.
_vscode_import.py	Pure transform for contextweaver mcp import-vscode (issue #367): build_migration_plan converts a VS Code-family MCP config's servers/mcpServers block into an upstreams: gateway config; render_gateway_config/render_replacement_config render the two output configs (kept separate from the plan since the replacement config embeds a write-time --gateway-config path decision); write_migration applies with a default backup. Not public API.
_demos.py	Demo logic for the CLI demo subcommand (exempt from print() rule)
serde.py	Serialisation helpers for to_dict / from_dict
tokens.py	Built-in token counter (count(), get_token_counter(), heuristic_counter(), TokenCounter alias) plus the provider-estimator registry (register_estimator(), registered_estimators(), estimator_name(), issue #493). The single source of truth for token counts — firewall, sensitivity-redaction placeholders, card budgeting (routing/cards.count_tokens), and FirewallStats/BuildStats numbers all route through it (issues #405/#493/#530); no stray len // 4 literals elsewhere. Owns the tiktoken dependency; offline it falls back to the script-aware HeuristicEstimator.
store/	In-memory data stores: EventLog, ArtifactStore, EpisodicStore, FactStore, StoreBundle
store/_sqlite_base.py	Shared SQLite connection + migration scaffolding (WAL, foreign_keys=ON, _contextweaver_schema_version table). Reused by every SQLite-backed store (issue #174).
store/sqlite_event_log.py	SqliteEventLog — first persistent EventLog backend; single-process, sync, append-only, schema-versioned (issue #223).
store/sqlite_episodic.py	SqliteEpisodicStore — persistent EpisodicStore on _sqlite_base (issue #496). Append-only, ordered by ordinal; search delegates to a transient InMemoryEpisodicStore for byte-identical ranking. Own version table (VERSION_TABLE) so it can share a DB file with the event log / facts.
store/sqlite_facts.py	SqliteFactStore — persistent FactStore on _sqlite_base (issue #496). put upserts on fact_id; get_by_key/all sorted by fact_id. Own version table; shareable DB file.
store/redis_artifacts.py	RedisArtifactStore — Redis ArtifactStore for multi-process gateways (issue #426). Namespaced keys, optional per-artifact TTL, list_refs via SCAN. Lazy redis import ([redis] extra).
store/redis_event_log.py	RedisEventLog — Redis EventLog (issue #426). Items in a hash keyed by id + parallel order list; append-only ordering across processes. Lazy redis import ([redis] extra).
store/s3_artifacts.py	S3ArtifactStore — S3-compatible ArtifactStore (issue #426; AWS/MinIO/R2/GCS). {prefix}/{handle}.data + .json objects. Lazy boto3 import ([s3] extra).
store/json_file_artifacts.py	JsonFileArtifactStore — filesystem ArtifactStore backend; {enc(handle)}.data + {enc(handle)}.json per artifact, re-instantiable against an existing directory (issue #42). Hardened (issue #497): atomic writes (temp file + os.replace), an in-memory handle→ref index built once on init so list_refs never rescans the directory, and optional max_bytes / max_artifacts quotas raising ArtifactStoreQuotaError. Persists content_hash and percent-encodes handles into filenames so the firewall's artifact:result:… handles are Windows-safe (issue #466). Optional ttl_seconds / redact_secrets lifecycle policy (issue #375) delegates to store/_json_file_ttl.py's ArtifactLifecycle.
store/_json_file_ttl.py	Private TTL + redaction-before-store bookkeeping for json_file_artifacts.py (issue #375): ArtifactLifecycle (composed, not inherited) — prepare() scrubs UTF-8 content via contextweaver.secrets.scrub_secrets before write, record_put()/is_expired()/forget() manage a process-lifetime-scoped expiry dict under an injectable monotonic Clock (mirrors gateway_controls.ToolResultCache's clock convention). Not persisted across a restart. Not public API.
store/_json_file_io.py	Private filesystem helpers for json_file_artifacts.py (keeps it ≤300 lines): on-disk suffix constants, validate_handle (path-traversal defense), encode_handle (percent-encoding), and the atomic_write primitive (issues #466/#497). Not public API.
store/testing.py	Store-protocol conformance kit (issue #520): framework-agnostic check_event_log_conformance / check_artifact_store_conformance / check_episodic_store_conformance / check_fact_store_conformance, each taking a factory for an empty backend and asserting the round-trip / ordering / not-found contract. No test-framework import; ships in the core wheel. tests/test_store_conformance.py runs every bundled backend through it.
summarize/	SummarizationRule, RuleEngine, extract_facts()
summarize/structured.py	Lossless JSON field projection for the firewall: parse_path / project + StructuredFirewall(keep=[...]). Deterministic, no LLM — keeps an allow-list of JSON paths inline and offloads the rest (issue #406).
context/	Full context pipeline, sensitivity enforcement, view registry, ContextManager
context/firewall_api.py	Single-call firewall facade: compact_tool_result / firewalled_tool_result → CompactResult. Composes structured/text strategies, schema-preserving pass-through (reserved _cw sidecar — a caller payload already using _cw raises ConfigError unless overwrite_sidecar=True, #467), the built-in token counter, and fail-closed deterministic mode (issues #399, #402, #403, #404, #405, #406, #467).
context/manager.py	ContextManager — thin orchestrator (__init__, properties, drilldown, mixin composition). Public method stubs live in flat partial-class mixins; pipeline logic lives in the delegate modules below (issue #101).
context/_manager_base.py	_ManagerState — private-attribute + _build contract the manager mixins inherit and the delegate pipeline modules type their manager parameter against (ContextManager inherits it via the mixins). Not public API (issue #101).
context/_manager_ingest.py / _manager_build.py / _manager_routing.py	_IngestMixin / _BuildMixin / _RoutingMixin — partial-class mixins holding ContextManager's ingestion, build, and route/call-prompt method surface as thin delegations; keep manager.py ≤300 lines (issue #101). Not public API.
context/ingest.py	Tool-result ingestion helpers (extracted from manager.py to honor the <=300 line guideline). Includes ingest_envelope — the canonical Frame-shaped seam (weaver-spec I-05) that ingests an already-firewalled ResultEnvelope without re-deriving firewalling; raw-output ingest_tool_result / ingest_mcp_result are non-canonical for spec compliance (issue #352).
context/memory_types.py	MemoryEntry dataclass + PHASE_SCOPE_PREFERENCES constants for phase-aware memory ingestion (issue #293).
context/memory_fixture.py	JsonFixtureMemorySource — deterministic stdlib fixture adapter implementing the MemorySource Protocol from protocols.py (issue #293).
context/memory_source.py	memory_entries_to_context_items / select_memory_for_phase helpers that materialise memory entries into budgeted memory_fact candidates (issue #293).
context/handoff_types.py	HandoffEntry + SessionHandoffPack dataclasses and canonical handoff category constants (issue #294).
context/handoff.py	build_session_handoff_pack / render_handoff_pack — deterministic, budget-aware, sensitivity- and firewall-respecting session continuity snapshot (issue #294).
context/advisor_pack.py	Advice-only planning escalation (issue #741): AdvisorRequest / AdvisorResponse + ask_advisor package a bounded, budget-truncated, strict-JSON prompt to a stronger model via call_fn (optionally GuardedCallFn-wrapped). Off-list preferred options are nulled with a deterministic marker; malformed output degrades to raw-text advice. Advice never executes or authorizes anything.
context/consolidation_types.py	ConsolidationPolicy / EpisodeCluster / PromotedFact / ConsolidationReport (+ CONSOLIDATION_REPORT_VERSION) — pure-data config and result types for the memory consolidation engine (issue #498).
context/consolidation.py	Memory consolidation engine (issue #498): cluster_episodes (deterministic episodic clustering/dedupe, #679), promote_clusters (fact promotion with provenance + max-sensitivity inheritance, #680; optional fail-closed call_fn merge, #682), decay_episodes / decay_facts (report-only decay over append-only stores, #681), and the consolidate(...) orchestrator → ConsolidationReport. Deterministic; apply=True upserts content-addressed facts (idempotent). Standalone functions (not a ContextManager method) mirroring handoff.py.
context/_consolidation_helpers.py	Private deterministic helpers for consolidation.py (clustering canonical text, max-sensitivity, session counting, ISO-timestamp parsing, content-addressed fact IDs, decay predicate) — keeps consolidation.py ≤300 lines. Not public API.
context/_consolidation_merge.py	Private optional model-assisted canonicalizer for consolidation (issue #682): refine_canonical_text runs a user-supplied call_fn under fail-closed guardrails (no LLM SDK dep; rejects blank/ungrounded completions that introduce tokens absent from the source cluster, falling back to the deterministic text). Not public API.
context/explanation.py	ContextBuildExplanation + CandidateExplanation opt-in debug surface returned by ContextManager.build(..., explain=True) (issue #291); carries resolved_weights (the per-phase scoring weights applied, issue #487). Sister to routing/explanation.py on the routing side.
context/build_policy.py	Pure build-pipeline policy helpers (not public API): override_phase_budget / adjust_budget_for_header (budget math), enforce_overflow_policy (ContextPolicy.overflow_action, issue #510), and render_pack_prompt (caller-owned renderer hook, issue #410). Extracted from build.py to keep it within its size ceiling.
context/classify.py	Opt-in deterministic ingestion-time sensitivity classification (issue #542): HeuristicSensitivityClassifier (implements the SensitivityClassifier protocol) + detect_sensitivity(). Runs at the start of the pipeline's sensitivity stage and over fact/episode header content; may only raise a label, never lower it. Reuses secrets.contains_secret plus PII markers.
context/secret_redaction.py	Opt-in SecretRedactor RedactionHook (issue #428): substring-scrubs secret shapes from an item's text via secrets.scrub_secrets. Registered under the name "secret" for ContextPolicy.redaction_hooks; complements (does not replace) MaskRedactionHook.
routing/	Catalog, ChoiceGraph, TreeBuilder, Router (beam search), card renderer
routing/filters.py	Pre-scoring helpers: filter_items(), augment_query(), suggest_clarifying_question() (issues #14, #22, #112, #116)
routing/catalog_metadata.py	Catalog inventory/governance metadata (issue #377): InventoryMetadata (owner/domain/risk/side-effects/lifecycle/environment/tier, all default None = unknown) under the reserved metadata["_contextweaver"]["inventory"] namespace via attach_inventory / inventory_of / validate_inventory / is_deprecated. Round-trips through SelectableItem serde untouched.
routing/collision.py	Collision & duplicate-capability analyzer (issue #381): analyze_collisions → CollisionReport (exact/near-name, description- and schema-similar findings + recommendation heuristic + render_markdown). Similarity via _utils + stdlib difflib; small schemas skipped for false-positive suppression.
routing/catalog_diff.py	Catalog diff + routing impact (issue #514): diff_catalogs (added/removed/field-level changes anchored by compute_catalog_hash), routing_impact (top-1 flips + recall@k over probe queries), suggest_probes heuristic probe generator. Backs catalog diff.
routing/vector_index.py	Embedding index over tool catalogs (issue #387): canonical_tool_text + VectorIndex (build / hash-based incremental refresh / query with per-section evidence / cosine duplicates). Backend-agnostic over EmbeddingBackend; deterministic with HashingEmbeddingBackend; plain-list vectors.
routing/export.py	Provider-native tool exporters for routed shortlists (issue #609): to_openai_tools / to_anthropic_tools / to_gemini_function_declarations → ExportedTools (wire-ready tools + deterministic sanitised-name→tool_id map with collision handling). Hydrates empty inline schemas via Catalog.hydrate (inline wins).
routing/manifest.py	GraphManifest + compute_catalog_hash() for graph metadata and cache invalidation (issue #48, #15)
routing/normalizer.py	CatalogNormalizer + NormalizationReport for catalog metadata hygiene (issue #44)
routing/catalog.py (validation)	validate_references / Catalog.validate_references → CatalogValidationReport of dangling depends_on/requires refs; loaders take on_invalid ("warn"/"raise"/"ignore") and raise CatalogValidationError in raise mode (issue #519)
routing/registry.py	EngineRegistry and bundled TfIdfRetriever / NoOpReranker / JaccardClusteringEngine defaults (issue #47)
routing/index_cache.py	Persistent, reusable fitted-index cache (issues #543/#624/#685): RoutingIndexCache (in-process LRU + optional deterministic-JSON on-disk layer) and CachedRetriever (a Retriever wrapper that loads/stores the fitted index keyed by a corpus fingerprint, transparently — warm loads score byte-identically to a cold fit). Pass via Router(retriever=CachedRetriever(TfIdfRetriever(), cache)). Codec + fingerprint live in routing/_index_codec.py.
routing/_index_codec.py	Private helper for index_cache.py: index_fingerprint() (deterministic ordered-corpus SHA-256) + the IndexCodec contract and bundled TFIDF_CODEC. Not public API; names re-exported from index_cache.
routing/trace.py	RouteTrace + TraceStep structured routing audit (issue #51)
routing/explanation.py	RouteResult.explanation() Markdown / dict rendering (issue #226)
routing/pipeline.py	RoutingPipeline composer — explicit retrieve → rerank → navigate → pack stages (issue #56)
routing/navigator.py	BeamSearchNavigator (lifted from router.py) + rank_collected — the score-sort/active-filter helper is re-exported from routing/__init__.py for custom Navigator implementations (issues #56, #288)
routing/packer.py	DefaultCardPacker wrapping make_choice_cards for the pipeline pack stage (issue #56)
routing/history.py	RouteHistory dataclass + adjust_scores (history-aware re-routing, issue #27)
routing/feedback.py	Optional feedback-aware routing scores (issue #318): ExecutionFeedback (contextweaver-native, not a weaver-spec type), DeterministicScoreProvider (default no-op), FeedbackAwareScoreProvider, aggregate_feedback. Plugs into Router(score_provider=...); default None keeps routing deterministic.
routing/selection.py	Structured route→select contract (issues #515/#479), both pure/deterministic: selection_schema emits the routed candidate IDs as a provider-native constrained-selection schema (json_schema/openai/anthropic) so a model can only pick a routed tool_id ("constrain before"); validate_selection → SelectionValidation validates/repairs (strip → case-fold → unique-prefix; ambiguous matches rejected, never guessed) a returned ID against the candidates ("validate after"). Surfaced on RouteResult.selection_schema() / RouteResult.validate_selection(); to_routing_decision resolves + records the outcome. Shortlist composition (pin_ids always-include + per-namespace namespace_quota, issue #509) lives in routing/filters.compose_shortlist and is exposed via Router.route(...).
extras/embeddings.py	SentenceTransformerBackend + HybridEmbeddingRetriever + HashingEmbeddingBackend (re-exported) behind the [embeddings] extra (issue #8)
extras/embeddings_hashing.py	HashingEmbeddingBackend — stdlib-only deterministic EmbeddingBackend using blake2b hashing trick; no extras required (issue #266)
_schema_gen.py	Dataclass → JSON Schema (Draft 2020-12) generator + make schemas-check engine (issue #225)
routing/tool_id.py	Canonical tool_id grammar (parse_tool_id / format_tool_id / compute_hash8) per docs/gateway_spec.md §1
routing/primitive_id.py	Unified cross-primitive identity + collision policy for tools/resources/prompts (parse_primitive_id / format_primitive_id / canonical_resource_id / canonical_prompt_id / resolve_collisions) per docs/gateway_spec.md §9. Tools keep the bare tool_id; resources/prompts get disjoint kind:: ids (issue #671).
routing/path.py	tool_browse path-navigation grammar (parse_path / resolve_path) per docs/gateway_spec.md §3
routing/hydration.py	Public schema-hydration helpers — SchemaSource (from raw dict / JSON file / MCP tools-list), hydrate_with_schema, lazy_schema_resolver. Reference architectures use these to resolve a tool's full input schema from a sidecar source rather than hand-rolling a _FULL_SCHEMAS dict. Inline args_schema on the catalog item wins; sidecar only fills empties. Issue #261.
adapters/	MCP, FastMCP, A2A, weaver-spec, CrewAI, Pydantic AI, smolagents, Agno, LangChain, OpenAI Agents SDK, Google ADK, Microsoft Agent Framework, OpenAPI, Agent Skills protocol adapters + MCP proxy / gateway runtime + provider-message ingestion helpers for OpenAI / Anthropic / Gemini chat histories. Framework tool-catalog adapters share adapters/_framework_common.py (issue #454). (issues #13, #28, #29, #34, #193, #194, #219, #222, #272, #274, #275, #430, #454, #501, #502, #545, #546, #547)
adapters/chainweaver.py	ChainWeaver flow-export → SelectableItem(kind="flow") import (chainweaver_flow_to_selectable, chainweaver_flows_to_catalog, load_chainweaver_export, issue #334). Pure data; no ChainWeaver dependency. Preserves name/description/input+output schemas; stamps metadata["runtime"]="chainweaver" + flow id/version.
adapters/crewai.py	CrewAI BaseTool (or equivalent plain-dict shape) ↔ SelectableItem (crewai_tool_to_selectable, crewai_tools_to_catalog, infer_crewai_namespace, load_crewai_catalog, issue #193)
adapters/pydantic_ai.py	Pydantic AI Tool ↔ SelectableItem and ModelMessage ↔ ContextItem lossless round-trip (pydantic_ai_tool_to_selectable, pydantic_ai_tools_to_catalog, load_pydantic_ai_catalog, from_/to_pydantic_ai_messages, issue #272) — heavy decode/encode helpers live in adapters/_pydantic_ai_messages.py
adapters/smolagents.py	Hugging Face smolagents Tool ↔ SelectableItem and MultiStepAgent.memory.steps → ContextItems (smolagents_tool_to_selectable, smolagents_tools_to_catalog, load_smolagents_catalog, from_smolagents_agent, issue #274)
adapters/agno.py	Agno (formerly Phidata) Function / Toolkit ↔ SelectableItem and AgentSession → ContextItems (agno_tool_to_selectable, agno_tools_to_catalog, load_agno_catalog, from_agno_session, issue #275)
adapters/_framework_common.py	Shared, framework-agnostic conversion scaffolding for the framework tool-catalog adapters (issue #454): infer_namespace, strip_namespace_prefix, coerce_schema_dict, collect_tags, require_name_description. Pure/stateless, imports no framework lib. Private — not exported. New adapters reuse these instead of re-implementing namespace/schema/tag mechanics.
adapters/langchain.py	LangChain BaseTool (or equivalent plain-dict shape) ↔ SelectableItem (langchain_tool_to_selectable, langchain_tools_to_catalog, infer_langchain_namespace, load_langchain_catalog, issue #502). [langchain] extra for live loading; plain-dict path needs no extra.
adapters/openai_agents.py	OpenAI Agents SDK FunctionTool ↔ SelectableItem and run items → ContextItems (openai_agents_tool_to_selectable, openai_agents_tools_to_catalog, load_openai_agents_catalog, from_openai_agents_run, issue #501). Run-item ingestion lives in adapters/_openai_agents_run.py. [openai-agents] extra for live loading.
adapters/google_adk.py	Google ADK tools ↔ SelectableItem and Session.events → ContextItems (google_adk_tool_to_selectable, google_adk_tools_to_catalog, load_google_adk_catalog, from_google_adk_session, issue #547). Session ingestion lives in adapters/_google_adk_session.py. [google-adk] extra for live loading.
adapters/agent_framework.py	Microsoft Agent Framework (AutoGen / Semantic Kernel lineage) tools ↔ SelectableItem and thread ChatMessages → ContextItems (agent_framework_tool_to_selectable, agent_framework_tools_to_catalog, load_agent_framework_catalog, from_agent_framework_thread, issue #430). Thread ingestion lives in adapters/_agent_framework_thread.py. [agent-framework] extra for live loading.
adapters/openapi.py	OpenAPI 3.0/3.1 operations → SelectableItem catalog (openapi_operation_to_selectable, openapi_spec_to_catalog, load_openapi_catalog, infer_openapi_namespace, issue #546). Routes over REST APIs; never calls them. Local $ref resolution + parameters/requestBody → args_schema composition + method→safety tags live in adapters/_openapi_schema.py. No extra — PyYAML/jsonschema are core.
adapters/agent_skills.py	Agent Skills (SKILL.md) directories → kind="skill" SelectableItems with lazy body hydration (skill_to_selectable, load_skills_catalog, parse_skill_frontmatter, SkillBodySource, issue #545). Frontmatter routes; SkillBodySource resolves the body/resources on selection (mirrors routing/hydration.SchemaSource). No extra — PyYAML is core.
adapters/_okf_io.py / _okf_materialize.py / _okf_coerce.py	Private OKF-style Markdown-plus-YAML-frontmatter parsing core shared by the four knowledge-source adapters below (issues #736/#763/#767/#776): permissive frontmatter parsing (never raises — degrades to a LoadDiagnostic), the KnowledgeNode dataclass, deterministic directory walk, and materialisation into ContextItem (kind=doc_snippet, provenance under the _contextweaver metadata namespace) + deterministic relevance scoring. _okf_coerce.py holds the pure frontmatter value-coercion helpers (scalar/float/expiry coercion + JSON-safe normalisation of dates/bytes, so the to_dict/ContextItem.metadata "JSON-compatible" contract holds), split out to keep _okf_io.py within the ≤300-line ceiling. Not public API; not re-exported at the adapters package level (mirrors mcp_primitives/gateway_primitives — a related family kept at submodule level to stay within adapters/__init__.py's frozen size ceiling).
adapters/okf.py	OKF bundle loader as a context source (issue #736): load_okf_bundle, okf_nodes_to_context_items, select_knowledge. index.md/log.md are bundle metadata/history, excluded from concept content by default.
adapters/repo_knowledge.py	Repository-knowledge bundles as context sources (issue #763): load_repo_knowledge (plain-Markdown fallback, max_files/max_total_bytes guardrails, links never auto-followed), classify_usage (deterministic usage tags — plain metadata strings, not Phase values; extending Phase is issue #587's separate concern), select_repo_knowledge.
adapters/lessons.py	LessonWeaver lesson bundles as lifecycle-aware context sources (issue #767): load_lesson_bundle, LessonSelectionPolicy (excludes rejected/deprecated/unreviewed-candidate lessons by default), eligible_lessons, select_lessons — every exclusion is reported with a reason, never silent.
adapters/expertise_pack.py	ExpertisePacks as bounded context sources (issue #776): load_expertise_pack validates pack structure (an index.md declaring version, every node carrying a key) — the canonical schema is tracked externally at dgenio/weaver-spec#184 and not yet bound. detect_conflicts is deterministic-only (literal same-key text disagreement, scoped by task_tags/expiry) — no LLM-backed contradiction inference.
adapters/proxy_runtime.py	ProxyRuntime shared core + ExposureMode enum + UpstreamCall Protocol (issue #29)
adapters/gateway_diagnostics.py / gateway_catalog_diagnostics.py	Sanitized ProxyRuntime instrumentation plus exact gateway/proxy static-schema exposure calculations: catalog, browse/hydrate/execute/view events, savings, artifact-view usage, and latency (issues #370/#378).
adapters/mcp_gateway.py	Two-tool gateway dispatch (tool_browse + tool_execute + tool_view, issues #28 / #34)
adapters/mcp_proxy.py	Transparent proxy dispatch (stripped tools/list + tool_hydrate + tool_execute, issue #13)
adapters/mcp_upstream.py	Concrete UpstreamCall adapters (StubUpstream, McpClientUpstream, MultiplexUpstream)
adapters/_config_coerce.py	Private config-value coercion helpers (interpolate_env for ${env:VAR}, coerce_bool, opt_positive_number, str_tuple, str_map) shared by upstream_config.py / startup_policy.py / artifact_policy.py (issue #366). Not public API.
adapters/upstream_config.py	Pure-data config for one live upstream MCP server (issues #366/#368): UpstreamSpec (type: stdio|http|sse, command/args/env or url/headers, namespace, required, include_tools/exclude_tools globs, timeout), parse_upstreams_config.
adapters/startup_policy.py	Fault-tolerant multi-upstream startup policy (issue #374): StartupPolicy (mode: degraded|strict, upstream_timeout_seconds, min_healthy_upstreams, fail_on_empty_catalog), UpstreamStatus, StartupReport, detect_tool_name_collisions (diagnostic only — routing still resolves collisions first-registered-wins via MultiplexUpstream).
adapters/artifact_policy.py	Artifact lifecycle policy for mcp serve --state-dir (issue #375): ArtifactPolicy (ttl_seconds, max_bytes, max_artifacts, redact_secrets), wired into JsonFileArtifactStore.
adapters/upstream_launch.py	Live multi-upstream launch behaviour (issues #366/#368/#374): launch_upstreams connects every configured UpstreamSpec under AsyncExitStack, classifies failures via gateway_error.classify_upstream_exception, and composes survivors behind MultiplexUpstream; NamespacedFilteredUpstream applies namespace-prefixing + include/exclude filtering at the list_tools boundary. Connect-step timeouts bound only session.initialize() (a self-contained RPC) — never the stdio_client/ClientSession context-manager entry itself, since wrapping that in asyncio.wait_for schedules a new Task and violates anyio's same-task cancel-scope invariant when AsyncExitStack closes it later from the caller's task.
adapters/mcp_gateway_server.py	Bind mcp_gateway onto mcp.server.Server over stdio (issue #28); optional primitive_runtime= also advertises/dispatches the four resource/prompt meta-tools (issues #669/#670)
adapters/mcp_primitives.py	MCP resource/prompt → SelectableItem(kind="resource"/"prompt") converters + resources/read / prompts/get result→envelope wrappers (issues #669/#670). Emits ids via routing/primitive_id.
adapters/gateway_primitives.py	PrimitiveGatewayRuntime + PrimitiveUpstream Protocol — bounded-choice routing + firewall for resources/prompts, sharing the tool runtime's ContextManager (issues #669/#670/#555).
adapters/_primitive_index.py	Private single-kind catalog+graph+router+browse helper for gateway_primitives (keeps it ≤300 lines). Not public API.
adapters/mcp_gateway_primitives.py	The four resource/prompt gateway meta-tools (resource_browse / resource_read / prompt_browse / prompt_get) + dispatch, mirroring mcp_gateway (issues #669/#670).
adapters/mcp_primitive_upstream.py	Concrete PrimitiveUpstream adapters mirroring mcp_upstream: StubPrimitiveUpstream (in-process), McpClientPrimitiveUpstream (wraps an MCP ClientSession), MultiplexPrimitiveUpstream (multi-server fan-out). Transport errors raise (the runtime classifies them) per the Protocol contract (issues #669/#670).
adapters/mcp_proxy_server.py	Bind mcp_proxy onto mcp.server.Server over stdio (issue #13)
adapters/sidecar_contract.py	HTTP sidecar wire contract (issue #674): RouteRequest/RouteResponse/CompactRequest/CompactResponse/SidecarError dataclasses + SIDECAR_API_VERSION. Pure, dependency-free; the published JSON Schemas live under schemas/sidecar/v1/.
adapters/sidecar.py	HTTP sidecar runtime (issue #675/#676): SidecarConfig + SidecarApp.dispatch — transport-free (method, path, headers, body) → (status, json) over the sync Router (/v1/route) and compact_tool_result facade (/v1/compact). Optional bearer-token auth, per-client rate limiting (reuses gateway_controls.RateLimiter), body-size cap, and typed SidecarError responses; never raises across the HTTP boundary.
adapters/_sidecar_http.py	Stdlib http.server.ThreadingHTTPServer binding for SidecarApp (issue #675). No third-party dependency. Public re-exports: serve_api (blocking serve) + make_sidecar_server (build-only, for tests). Not public API itself.
adapters/_sidecar_validation.py	Stateless parsing + field-validation helpers shared by sidecar.py and sidecar_contract.py (request-body JSON decode, bearer-token extraction, typed contract-field coercions). Pure, dependency-free; raises ConfigError on malformed input. Not public API.
adapters/gateway_error.py	Structured GatewayError (codes + §3.4 wire shape) + retryable hint. Upstream-error taxonomy: classify_upstream_exception maps timeouts/connection/auth/permission/rate failures to UPSTREAM_TIMEOUT/UPSTREAM_UNAVAILABLE/AUTH_FAILED/PERMISSION_DENIED/RATE_LIMITED (fallback UPSTREAM_ERROR); redact_upstream_detail strips control chars + caps length on model-visible detail (issue #485).
adapters/gateway_validation.py	Untrusted-schema hardening for the gateway ingest path (issues #464/#484): SchemaLimits/SchemaFinding/SkippedTool/CatalogRefreshReport, check_schema_health (meta-validation + iterative size/depth/property bounds), build_validator (cached per tool_id). Pure, deterministic; iterative traversal avoids stack exhaustion on hostile schemas.
adapters/gateway_args.py	Opt-in deterministic tool-call argument repair (issue #488): normalize_args (stringified-object parse + schema-demanded str→int/number/boolean/null coercion) + Repair. Gated behind ProxyRuntime(tolerant_args=True); never renames keys, drops keys, or fuzzy-matches.
adapters/gateway_policy.py	Pure-data config + result types for the dispatch-path controls (issues #529/#482/#483): RetryPolicy (bounded backoff), RateLimit/RateLimitPolicy (per-session quotas), DryRunReport. All defaults inert; to_dict/from_dict for mcp serve --config.
adapters/gateway_controls.py	Runtime mechanisms behind gateway_policy (issues #529/#482/#512): call_with_retry (retry loop, injectable sleep), RateLimiter (sliding-window + cumulative counters, injectable clock), ToolResultCache (TTL+LRU read-only response cache). All opt-in; wired into ProxyRuntime.execute/browse/view.
adapters/gateway_authz.py	Runtime authorization / policy gate (issue #373): ToolPolicy (ordered PolicyRules + default), PolicyContext, PolicyDecision, policy_gate_error. Pure/deterministic (first-match, case-sensitive globs). Evaluated before tool_execute dispatch and — via meta_tool="tool_view" rules — before tool_view raw egress (issue #746). Default allow (inert); deny→POLICY_DENIED, require_approval→AUTH_REQUIRED. to_dict/from_dict for the policy block of mcp serve --config.
adapters/gateway_visibility.py	Audience-scoped catalog visibility profiles (issue #379): VisibilityProfile (glob include/exclude over namespaces/domains, risk/lifecycle exclusion, side-effect + environment allowlists), evaluate_visibility/filter_catalog (denials carry reasons), parse_profiles for a visibility_profiles: config block. Mirrors gateway_authz's first-match pattern; exclude rules fail open on unknown inventory metadata, allowlists fail closed. Applied at serve startup before ChoiceCards.
adapters/catalog_pin.py	Catalog pinning against tool-surface drift (issue #656): PinPolicy (expected_hash, mode: warn|strict), check_catalog_pin via routing.manifest.compute_catalog_hash, enforce_pin (strict-mode mismatch raises with both hashes + re-pin hint). pin: block of mcp serve --config.
adapters/live_refresh.py	Notification-driven live catalog refresh (issue #424): LiveRefreshPolicy + LiveRefresher turn upstream tools/list_changed into ProxyRuntime's atomic register_tool_defs_sync — debounced, sliding-minute rate-limited, inert by default; make_message_handler wires it to the ClientSession message handler threaded through launch_upstreams.
adapters/serve_lifecycle.py	Graceful shutdown for the serve loops (issue #626): ShutdownController — SIGINT/SIGTERM capture (graceful where loops lack signal support), drain-then-cancel of in-flight work, defensive flush/close of stores and sinks with per-object error collection → ShutdownReport. Transport-agnostic.
adapters/memory_server.py	Standalone MCP memory server (issue #632): build_memory_server / run_memory_server_stdio expose episodic/fact stores as four MCP tools (memory_add_episode/search_episodes/put_fact/get_facts) with deterministic content-hash ids, the gateway ARGS_INVALID error shape, and opt-in read-path secret scrubbing via the shared contextweaver.secrets helpers. Backs mcp memory-serve.
adapters/gateway_scorecard.py	Tool-surface health scorecard (issue #380): build_scorecard joins inventory metadata (#377) with diagnostic-event streams (#382) into a deterministic report (usage/latency/failure hot-spots, unused/deprecated-in-use, collision counts via #381) + Markdown/CSV/JSON renders. Backs mcp scorecard.
adapters/gateway_status.py	Gateway status surface for mcp status (issue #655): GatewayStatus snapshot + rate-limited atomic StatusWriter (injectable clock, coalescing, force()), read_status (ConfigError with --state-dir hint), render_status (uptime + 30s staleness warning).
adapters/gateway_doctor.py / _doctor_checks.py	Gateway preflight for mcp doctor (issue #395): run_doctor → DoctorReport of never-raising DoctorFindings — config/key-set/XOR checks, catalog load + references + weak-metadata, ChoiceCard schema-hiding probe, hydration, artifact-store writability, optional-extras info, opt-in live upstream launch + routing smoke queries. Check bodies live in the private _doctor_checks.py (size split).
adapters/sampling_call_fn.py	MCP sampling-backed call_fn (issue #623): make_sampling_call_fn / make_async_sampling_call_fn bridge the firewall's Summarizer seam to the connected client's model via sampling/createMessage. Sync variant runs on a worker thread through the server loop with a deadlock guard + timeout; non-text results raise so consumers fall back to the deterministic path. Opt-in only.
adapters/wire_capture.py	Record/replay harness pinning gateway protocol behaviour with committed golden transcripts (issue #654): WireRecorder / save_transcript / load_transcript / replay_and_verify at the dispatch boundary (JSON-RPC framing is SDK-owned; content bodies JSON-decoded for field-level diffs, volatile paths normalised).
adapters/gateway_presets.py	Named gateway policy presets (issue #664): GatewayPreset.from_preset("safe"|"balanced"|"throughput") bundles a ToolPolicy + RetryPolicy + RateLimitPolicy + CacheConfig (also defined here — pure-data config for gateway_controls.ToolResultCache). Selectable via mcp serve --policy-preset / policy_preset config key; an explicit policy/retry/rate_limits/cache block wins over the preset for that block. Deterministic to_dict() backs mcp serve --print-effective-policy.
adapters/_proxy_dispatch.py	Private dispatch helpers keeping proxy_runtime.py within its size ceiling: persist_result_artifacts, execute_policy_error/view_policy_error (policy-context construction + gate call), rate_limited_error, unverified_annotations, build_dry_run_report, UpstreamNameIndex. Not public API.
adapters/openai_messages.py	OpenAI Chat Completions messages ↔ ContextItem round-trip (from_/to_openai_messages, issue #219)
adapters/anthropic_messages.py	Anthropic Messages API messages ↔ ContextItem round-trip (from_/to_anthropic_messages, issue #222)
adapters/gemini_contents.py	Google Gemini contents[] ↔ ContextItem round-trip (from_/to_gemini_contents, issue #222)
extras/otel.py	OpenTelemetry GenAI integration (OTelEventHook — invoke_agent / execute_tool spans + GenAI SemConv attributes, gated behind the [otel] extra, issue #224).
extras/llm_summarizer.py	Optional LlmSummarizer / LlmExtractor — LLM-backed Summarizer / Extractor plugins for the firewall. Take a user-supplied call_fn (no LLM SDK dep, no extra) and degrade to the rule-based path on any failure (issue #26). Optional provider_metadata surfaces on FirewallStats.llm_provider for audit; truncated inputs carry a deterministic omission marker (issue #384).
extras/llm_guard.py	GuardedCallFn — policy envelope for user-supplied LLM call_fn callables (issue #494): GuardPolicy (call cap, consecutive-failure circuit breaker with cooldown/half-open trial, timeout accounting — post-hoc by default, opt-in thread-enforced hard timeout) + live GuardStats. Injectable monotonic clock. Rejections raise PolicyViolationError so plugins degrade to their deterministic fallbacks. No LLM SDK dep.
extras/catalog_enrich.py	Offline LLM-assisted catalog metadata enrichment (issue #383): enrich_catalog emits reviewable EnrichmentSuggestions from metadata-only prompts (enum values validated against #377 vocabularies; malformed output + guard rejections collected as skips), never mutating the catalog; apply_suggestions is the explicit reviewed opt-in.
extras/ranker.py / _ranker_examples.py	Telemetry-trained tool reranker behind the [ranker] extra (issue #388): deterministic featurize, sklearn-gated ToolRanker (fixed-seed fit, JSON coefficient + model-card persistence — no pickle, pure-Python prediction so saved models load without sklearn), evaluate_ranker vs a lexical baseline. Example derivation from DiagnosticEvents in the private sibling.
extras/model_setup.py	Local embedding-model setup helpers behind models doctor / models download (issue #386): EmbeddingModelConfig (the models.embeddings block), network-free run_model_doctor (extra/torch/cache/model-presence checks with install hints), and an explicit download_model (the only network-touching call).
extras/memory/	External-memory backend adapters that implement EpisodicStore / FactStore against an existing long-lived memory deployment without widening the Protocols (issue #195).
extras/memory/mem0.py	Mem0EpisodicStore + Mem0FactStore — wrap a mem0.Memory instance scoped by user_id; writes go through Memory.add(infer=False) and items are stamped with cw_episode_id / cw_fact_id metadata for canonical-ID resolution. Gated behind the [mem0] extra (issue #195).
extras/memory/zep.py	ZepEpisodicStore + ZepFactStore — wrap a zep_cloud.Zep client scoped by user_id; persist items as JSON graph episodes (graph.add(type="json")) stamped with cw_* IDs, resolving back via graph.episode.get_by_user_id. Episodic search is client-side (Zep graph search is edge/node-shaped). Gated behind the [zep] extra (issue #195).
extras/memory/_zep_common.py	Internal helpers backing zep.py (keeps it ≤300 lines): shared cw_* constants, the ZepBackendError exception, the JSON/scan helpers (_episode_records / _episode_uuid / _episode_payload), the defensive payload-coercion helpers (_coerce_str_tags / _coerce_metadata), and the _ZepStoreBase scope/scan/write base. Carries the same [zep]-extra import guard (issue #195).
extras/memory/langmem.py	LangMemEpisodicStore + LangMemFactStore — wrap any LangGraph BaseStore scoped by a namespace tuple; canonical ID is the store key, value is the dataclass to_dict() payload (direct, lossless KV). search delegates to BaseStore.search. Gated behind the [langmem] extra (issue #195).
eval/	Evaluation harness (issue #12): EvalCase / EvalDataset (gold datasets), evaluate_routing → RoutingEvalReport (top-k recall, MRR, confidence gap, beam steps), evaluate_context → ContextEvalReport (budget utilisation + token savings vs naive concat). Pure-stdlib, deterministic; backs the eval CLI subcommand.
eval/consolidation.py	Consolidation quality evaluation harness (issue #683): evaluate_consolidation → ConsolidationEvalReport (precision / coverage against an optional gold set + dedup ratio). Pure-stdlib, offline, deterministic.
eval/metrics.py	Canonical rank-based routing metrics — recall_at_k (classic fractional recall@k), precision_at_k, reciprocal_rank (issue #354). Single source of truth imported by both eval/routing.py and benchmarks/benchmark.py so the harness and the benchmark script can no longer define the same names with different semantics.
__main__.py	CLI: 15 top-level subcommands (start, demo, build, route, print-tree, init, ingest, replay, stats, inspect, budget-check, eval, consolidate, verify, serve-api) plus the mcp and catalog Typer sub-apps. start prints deterministic deployment-intent guidance without side effects (issue #660); inspect renders payload-safe context/routing/artifact reports (issue #398); catalog lint surfaces normalization + reference findings with --json and CI exit codes (issue #538).
_mcp_cli.py	Backs the mcp Typer sub-app. Hosts mcp serve, mcp inspect, mcp stats, mcp generate-configs, mcp incident-pack, and mcp import-vscode; accepts native contextweaver, raw MCP tools/list, and {tools:[...]} catalog shapes. mcp serve --diagnostics FILE appends sanitized JSONL and --quiet suppresses lifecycle stderr; both are config-file keys. mcp serve --state-dir DIR (config key state_dir) persists gateway state — events.sqlite3 + artifacts/ — so artifact handles and event history survive a restart (issue #511); omit it for the in-memory default. An artifacts: config block (issue #375) wires TTL/quota/redaction into that store. mcp generate-configs emits deterministic multi-client recipe artifacts from one canonical mcp serve --config input (issue #659). mcp incident-pack creates local redacted triage bundles and never reads shell history automatically (issue #661). mcp serve --policy-preset (config key policy_preset) selects a named GatewayPreset (issue #664); an explicit policy/retry/rate_limits/cache block still wins over the preset for that block. mcp serve --print-effective-policy prints the resolved bundle as JSON and exits without requiring the catalog to exist on disk. The default serve path remains a static catalog + stub upstream; an upstreams: config block (issues #366/#368/#374) switches to _serve_live(), which launches real upstream MCP servers via adapters.upstream_launch.launch_upstreams under a startup: fault-tolerance policy — resources/prompts are not yet supported over live upstreams. mcp import-vscode (issue #367) migrates a VS Code MCP config into an upstreams: gateway config via _vscode_import.py; defaults to --dry-run.
data/	Packaged data files shipped inside the wheel via [tool.setuptools.package-data]. Exposes gateway_catalog_path() (resolves mcp_gateway_catalog.yaml to a concrete Path for both editable installs and zipped wheels — falls back to a persistent cache under tempfile.gettempdir()/contextweaver/ for zipimport). Issue #264.
examples/recipes/	MCP-client integration recipes: installed-CLI configs for Claude Desktop, Claude Code, GitHub Copilot, and Cursor plus gateway_config.yaml; serve_gateway.py remains a legacy/custom-runtime launcher (issues #278, #279, #346, #371, #429, #437).

Pipelines (summary)

Context Engine — 8 stages:

1. generate_candidates → 2. dependency_closure → 3. sensitivity_filter →
2. apply_firewall → 5. score_candidates → 6. deduplicate_candidates →
3. select_and_pack → 8. render_context

Routing Engine — 4 stages:

1. Catalog → 2. TreeBuilder → 3. Router (beam search) → 4. ChoiceCards

The Router itself composes a four-stage RoutingPipeline internally (retrieve → rerank → navigate → pack, issue #56). Each stage is swappable via the EngineRegistry or by passing a custom RoutingPipeline to Router(pipeline=...). History-aware re-routing (Router.route(history=...), issue #27) and the optional embedding retriever (Router(embedding_backend=...), issue #8) plug into this same pipeline contract.

For full pipeline descriptions and design rationale, see docs/agent-context/architecture.md.

Key Types

Type	Purpose
SelectableItem	Unified tool/agent/skill/flow/internal item (kind="flow" = external multi-step capability, e.g. a ChainWeaver flow). Deprecated alias: ToolCard (use SelectableItem in code; removal in 1.0 — see docs/upgrading.md).
ContextItem	Event log entry with parent_id for dependency closure
ResultEnvelope	Processed tool output: summary + facts + artifacts + views
ContextPack	Rendered prompt + stats from a context build
BuildStats	What was kept, dropped, and why. total_candidates is pre-sensitivity; dropped_items attributes every exclusion; completed builds satisfy included + dropped = total. Carries firewall_events + firewall_summary() (issues #402/#414/#459) and token_estimator — the estimator-path identifier that produced the build's token numbers (issue #493).
FirewallStats	Per-firewall diagnostics: triggered, strategy, original/summary chars+tokens, artifact_ref, summarized_by_llm (issues #402 / #404)
CompactResult	Output of the single-call compact_tool_result facade: firewalled, payload, summary, facts, artifact_ref, stats (issue #399)
StructuredFirewall	Non-summarising firewall strategy — keep an allow-list of JSON paths inline, offload the rest (issue #406)
ChoiceCard	LLM-friendly compact card (never includes full schemas)
RoutingDecision	Routing output shaped for weaver-spec interop (id, choice_cards, timestamp, selection). choice_cards is a flat list of CW 1:1 cards; for schema-valid spec JSON, go through adapters.weaver_contracts.to_weaver_routing_decision(). Build with RouteResult.to_routing_decision(...).
ChoiceGraph	Bounded DAG for routing, serializable, validated on load
GraphManifest	Build-time metadata attached to every routing graph (hash, seed, engine versions, timestamp)
RouteTrace	Always-populated structured audit of a routing call; per-step expansions opt-in via debug=True
EngineRegistry	Pluggable registry for Retriever, Reranker, ClusteringEngine slots
Mode	Determinism mode (strict / seeded / adaptive placeholder) on ProfileConfig
MaskRedactionHook	Built-in redaction hook for sensitivity enforcement
HydrationResult	Result of hydrating a tool call with context
ConsolidationReport	Deterministic result of a consolidate() run: episode clusters, promoted facts (with provenance + inherited sensitivity), and report-only decayed episode/fact IDs (issue #498)
ViewRegistry	Maps content-type patterns to view generators for progressive disclosure
ProxyRuntime	Shared core for MCP proxy (#13) and gateway (#28) modes — owns upstream catalog, per-session ContextManager, browse / execute / view dispatch; persisted text results are returned as envelope artifact refs for tool_view. Hardens the untrusted-input boundary (issues #464/#484/#485/#488): on_invalid (skip/raise) + schema_limits + last_refresh_report at ingest, cached per-tool_id validators, classified+redacted upstream errors, and opt-in tolerant_args. Opt-in dispatch-path controls (issues #529/#482/#512/#483): retry_policy, rate_limiter, result_cache, and tool_execute(dry_run=True) — all inert by default; catalog refresh rebuilds all derived state atomically (#507). Optional policy (ToolPolicy, issue #373) gates tool_execute dispatch and tool_view egress (#746); None by default (allow-all).
ExposureMode	TRANSPARENT (#13) vs GATEWAY (#28) for ProxyRuntime
UpstreamCall	Transport-agnostic Protocol over upstream MCP fan-out (used by ProxyRuntime)
PrimitiveGatewayRuntime	Resource/prompt counterpart to ProxyRuntime: bounded-choice browse + firewalled read/get over MCP resources and prompts, sharing the tool runtime's ContextManager (issues #669/#670)
PrimitiveUpstream	Transport-agnostic Protocol for upstream resources/list / resources/read / prompts/list / prompts/get (sibling of UpstreamCall)
GatewayError	Structured error payload (§3.4) returned from every gateway/proxy meta-tool. Carries a retryable hint and a classified upstream-error taxonomy (issue #485) plus the SCHEMA_INVALID ingest code (issue #484) and the POLICY_DENIED/AUTH_REQUIRED authorization codes (issue #373)
ToolPolicy	Runtime authorization gate (issue #373): ordered PolicyRules + default action (allow/deny/require_approval) evaluated before tool_execute dispatch and tool_view egress. Pure/deterministic; None on ProxyRuntime is allow-all.
GatewayPreset	Named policy preset (issue #664): bundles ToolPolicy + RetryPolicy + RateLimitPolicy + CacheConfig. from_preset("safe"|"balanced"|"throughput"); deterministic to_dict(). Selecting no preset is inert.
ToolIdParts	Destructured canonical tool_id (namespace / name / version / hash8)

Vocabulary notes:

• SelectableItem is the canonical name. ToolCard is a deprecated alias (documentation-only deprecation, scheduled for removal in 1.0 — see docs/upgrading.md) — use SelectableItem in code and docs.
• "Context" is overloaded — can mean ContextItem, ContextPack, the pipeline, or the LLM context window. Disambiguate when unclear. See docs/concepts.md.
• "Firewall" here means context firewall (prevents large outputs from consuming the token budget), not a security firewall.

Commands

make fmt      # ruff format src/ tests/ examples/ scripts/
make lint     # ruff check src/ tests/ examples/ scripts/
make type     # mypy src/ examples/ scripts/  (examples + scripts gated too, #539)
make test     # python -m pytest --cov=contextweaver --cov-report=term-missing -q
make example  # run all example scripts (includes architectures via the umbrella target)
make architectures  # run reference architecture scripts under examples/architectures/
make demo     # python -m contextweaver demo
make ci       # fmt + lint + type + test + drift-check + module-size-check + doc-snippets-check + readme-version-check + security-policy-check + example + demo
make docs     # mkdocs build --clean (docs site)
make docs-serve  # mkdocs serve (live preview)
make benchmark        # run benchmark harness (non-gating; writes benchmarks/results/latest.json)
make benchmark-matrix # benchmark + per-backend × per-size matrix (#208) and per-namespace breakdown (#209)
make benchmark-large-catalog  # 300+ tool routing/filter/firewall scorecard (#369); -check gates drift + floors
make benchmark-scenario       # ChoiceCards + ambiguity/firewall scenario report (#418); -check gates drift
make trend            # render benchmarks/trend.md from per-release history snapshots (#554)
make trend-check      # verify benchmarks/trend.md is up to date (exits non-zero on drift)
make gateway-scorecard-check  # verify gateway scorecard matches its committed JSON (gating CI; #391)
make record-demos-check       # verify committed demo casts match current output (gating CI; #390)
make smoke-eval       # gating CI smoke-evaluation over fixed fixtures (#331/#392/#491); deterministic, credential-free
make scorecard        # render benchmarks/scorecard.md from benchmarks/results/latest.json
make scorecard-check  # verify scorecard.md is up to date (exits non-zero on drift)
make schemas         # regenerate schemas/ + docs/schemas/v0/ (issue #225)
make schemas-check    # verify published schemas match dataclasses (gating, in `make ci`)
make drift            # regenerate every committed generated artifact (issue #522)
make drift-check      # one gate over all generated-artifact drift checks (in `make ci`, #522)
make api             # regenerate api/public_api.txt (public-API manifest, #518)
make api-check        # verify the public-API manifest matches the surface (in drift-check, #518)
make module-size-check # enforce the ≤300-line convention; frozen baseline (gating, #456)
make module-size-update # re-snapshot scripts/module_size_baseline.json (deliberate use only)
make doc-snippets-check # execute README + curated docs Python snippets (gating, #526)
make sweep-scoring    # weight sweep for ScoringConfig (#214); writes benchmarks/sweep_scoring.md
make context-rot       # render the context-rot demo: benchmarks/results/context_rot.json + docs/assets/context_rot.svg (#349)
make context-rot-check # verify context_rot.svg matches its committed JSON (gating in CI; exits non-zero on drift)
make readme-version-check  # verify README version references match pyproject.toml (gating in CI; #347)
make security-policy-check # verify SECURITY.md supported series + links match pyproject.toml (gating in CI; #691)
make llms        # regenerate llms.txt and llms-full.txt from canonical docs
make llms-check  # verify llms.txt and llms-full.txt are up to date (gating in CI; #389)
make weaver-conformance  # round-trip + JSON-Schema validate the weaver-spec adapter (CI gating, fetches schemas)

Run pre-commit install once after cloning to activate git hooks (ruff format + check + file hygiene on every commit).

For command-selection rules and sequencing, see docs/agent-context/workflows.md.

Hard Rules

These are auto-reject in review. No exceptions.

1. No print() in library code. Use hooks or logging. __main__.py and _demos.py (CLI) are exempt.
2. No business logic in __init__.py. Only re-exports allowed.

Strong Patterns

These are strongly recommended. Engineering judgment applies — deviate with good reason.

• Text similarity in _utils.py only — tokenize(), jaccard(), TfIdfScorer are the single source of truth. Do not duplicate.
• from __future__ import annotations in every source file.
• All exceptions from contextweaver.exceptions — use the custom hierarchy, not bare ValueError/RuntimeError. A new exception class needs a unique stable code, a GOLDEN_CODES entry in tests/test_exceptions.py, and a section in docs/errors.md.
• to_dict() / from_dict() on all dataclasses — complements serde.py; they are not redundant. See invariants.
• Deterministic by default — tie-break by ID, sorted keys. No randomness in core pipelines.
• No wildcard imports — never use from contextweaver import *.
• Event log is append-only — mutate only via InMemoryEventLog.append().

Coding Style

• Python ≥ 3.10 — use X | Y union syntax, match statements where appropriate.
• Type hints on all public functions and methods.
• Google-style docstrings on all public classes and functions.
• 100-character line length (enforced by ruff).
• ≤ 300 lines per module — exempt: types.py, envelope.py, __main__.py, _mcp_cli.py (experimental Typer sub-app; size is dominated by Typer parameter declarations + docstrings and is expected to shrink once mcp serve graduates from [experimental] to stable), and _demos.py (CLI demo-output module — print-heavy walkthrough scripts backing the demo subcommand, same rationale as __main__.py). Enforced mechanically by make module-size-check (issue #456): new non-exempt modules must stay ≤300 lines, and pre-existing oversized modules are grandfathered at their current size in scripts/module_size_baseline.json and frozen — they may shrink but may not grow past their recorded ceiling. Decomposing a grandfathered module lowers its ceiling via make module-size-update.
• Core runtime dependencies. The core install pulls tiktoken, PyYAML, rank-bm25, plus mcp and jsonschema (added when the proxy / gateway runtimes landed — both are load-bearing for docs/gateway_spec.md §4.4 schema validation and the MCP transport binding). Adding another core dependency requires explicit justification: broad ecosystem use, small wheel, and a default the library would otherwise have to approximate. Heavy or runtime-specific packages (CLI, OpenTelemetry, fuzzy retrieval, ANN, NetworkX, FastMCP, LangChain) live under [project.optional-dependencies] and are loaded via guarded imports.
• Dependency-constraint policy (issue #356). Specifiers are lower-bound-only (>=), set to the lowest version actually known to work — no == pins, no speculative upper caps. The only caps kept carry an inline rationale (pre-1.0 weaver_contracts<1; docs-extra major pins). Two CI jobs enforce this: a gating floor-deps job (uv pip install --resolution lowest-direct, Python 3.10, in ci.yml) proves the >= floors, and a non-gating weekly deps-latest-weekly.yml (latest + pre-releases) is the no-upper-cap safety net. Raising a floor is a real change — verify with the floor-deps job. Python support is 3.10–3.13, every cell gating in the CI matrix (3.14 is pending — the heavy dev/adapter stack still caps at Requires-Python <3.14).

Testing

• Tests in tests/test_<module>.py — one file per module.
• pytest.mark.asyncio for async tests (asyncio_mode = "auto" is set globally).
• Do not mock internal modules — use real in-memory implementations.

Path Conventions

store/ — Protocols are backend-agnostic (must not import backend-specific libraries). Concrete implementations may import backend libs. Must implement the protocol from protocols.py. Data is append-only / immutable-after-write.

adapters/ — Pure stateless converters. External format parsing must not leak into core. May import optional external libraries at the adapter boundary only.

context/ — Async-first. All new code should be async with _sync wrappers.

routing/ — Sync-only. Pure computation (DAG traversal, beam search). Do not make async.

Sensitivity (context/sensitivity.py) — Security-grade code. Extra review scrutiny required. Never weaken defaults. Treat changes like security-sensitive code. Redaction is effective end-to-end: MaskRedactionHook drops the item's artifact_ref and stamps metadata["redacted"]=True so the rendered prompt never advertises a handle that drilldown could dereference back to the original (issue #451). Enforcement is also applied on the prompt header (facts + episode summaries are routed through the floor and the memory_fact phase policy, issue #450) and an opt-in SensitivityClassifier may raise labels before enforcement (issue #542).

Things That Must Not Be "Simplified"

1. Protocol-based store design — the protocol layer exists for backend extensibility. Do not collapse protocols into concrete classes.
2. dependency_closure pipeline stage — if a selected item has parent_id, the parent must be included. Removing it produces incoherent context (tool results without their tool calls).
3. serde.py + per-class to_dict/from_dict — complementary, not redundant. serde.py provides shared primitives; per-class methods handle class-specific serialization. Do not consolidate.
4. ContextManager mixin composition — its public method surface lives in flat partial-class mixins (_IngestMixin / _BuildMixin / _RoutingMixin) sharing the _ManagerState base. Do not "simplify" this into delegating composition (manager.ingest.x()): that would change the public method surface, which issue #101 forbids ("ContextManager still exposes all current methods"). Mixins were the deliberate trade-off to keep manager.py ≤300 lines without breaking the public API.

See docs/agent-context/invariants.md for the full invariants list and rationale.

Debugging Tips

1. make lint — check for style and import errors.
2. make type — check for type errors.
3. make test — run the test suite.
4. Check BuildStats fields to understand what the context engine dropped and why.
5. Use ContextManager.artifact_store.list_refs() to inspect intercepted tool outputs.
6. Enable logging.DEBUG on contextweaver.context to trace pipeline stages (candidate counts, scores, drops, budget usage).
7. Enable logging.DEBUG on contextweaver.routing to trace beam search expansions and scoring.

Adding a Feature

1. Identify the relevant module, modify it, add tests in tests/test_<module>.py.
2. Run make ci to verify (all declared targets must pass).
3. Update CHANGELOG.md and add docstrings to new public APIs.
4. Update agent-facing docs and examples if the pipeline or public API changed.
5. If the feature can move recall@k / drops / dedup / token counts: follow .github/prompts/add-eval.prompt.md to extend the gold set or scenarios and run make benchmark-matrix && make scorecard. CI will post a sticky benchmark-delta comment on the PR.

For the full workflow and definition of done, see docs/agent-context/workflows.md.

Common Pitfalls

See docs/agent-context/lessons-learned.md for durable recurring mistakes and how to avoid them.

Documentation Map

File	Role
AGENTS.md (this file)	Primary shared source of truth for all agents
docs/agent-context/architecture.md	Non-obvious architectural guidance and tradeoffs
docs/agent-context/workflows.md	Authoritative commands, sequencing, definition of done
docs/agent-context/invariants.md	Hard constraints and forbidden shortcuts
docs/agent-context/lessons-learned.md	Failure-capture workflow and durable lessons
docs/agent-context/review-checklist.md	Self-check and review gates
docs/architecture.md	Canonical architecture reference (full pipeline detail, diagrams)
docs/concepts.md	Core concept glossary (types, subsystems, phases)
CONTRIBUTING.md	Human contributor guide

When architecture details conflict, docs/architecture.md is the canonical reference.

Update Policy

• Update AGENTS.md when shared rules, conventions, or the module map change.
• Update docs/agent-context/ files when their specific topic area changes.
• Any PR that changes the pipeline, public API, or project conventions must include doc updates.
• If two docs disagree, AGENTS.md is authoritative for agent guidance; docs/architecture.md is authoritative for architecture detail.
• See docs/agent-context/workflows.md for documentation governance rules.