Awesome Harness Engineering

This repository is a curated resource list for Harness Engineering and AI Native Engineering. Entries are filtered for direct relevance. GitHub projects include star counts at collection time.

Special Thanks to linux.do

Last researched: 2026-05-30 (Asia/Shanghai).

1) Articles

Harness Engineering Foundations

• Harness Engineering OpenAI's framing of harness engineering as the scaffolding layer around agents: context, tools, memory, verification, and execution boundaries.

• Unrolling the Codex Agent Loop Breaks the coding-agent loop into observe, plan, act, and verify stages, making harness responsibilities explicit.

• Building Effective Agents Anthropic's practical guide to choosing workflows, agents, and multi-step compositions for production systems.

• Harness Design for Long-Running Application Development Focuses on how to preserve progress, state, and reliability across long-running or multi-session agent tasks.

• Writing Effective Tools for Agents Treats tool schemas, error surfaces, and naming as first-class agent UX and harness design.

• Harness Engineering Martin Fowler's synthesis of context engineering, architectural constraints, and entropy management as a single engineering discipline.

• The Anatomy of an Agent Harness Organizes harness design into filesystem, code execution, sandboxing, memory, and context management primitives.

• Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineering A fresh systems review of why agent capability is increasingly externalized into memory, skills, protocols, and harnesses rather than model weights alone.

• AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents Formalizes harness as the runtime substrate between model and environment, with responsibilities spanning task specification, context selection, tools, memory, state, observability, failure attribution, verification, permissions, entropy auditing, and intervention records.

• Code as Agent Harness Frames code as the executable substrate for agent reasoning, action, environment modeling, memory, verification, and multi-agent coordination.

Harness / Platform Engineering in Practice

• Harness AI Overview Explains how Harness applies AI in CI/CD, release governance, and secure automation.

• Harness Agents Describes autonomous pipeline workers, RBAC boundaries, and integration patterns.

• AI Test Automation Covers AI-assisted testing workflows and how they reduce maintenance overhead.

• Harness Revamps IDP to Give Platform Engineers More Granular Controls Describes ownership and governance updates in Harness Internal Developer Portal.

• Harness engineering: Agent harnesses as critical infrastructure Enterprise-facing overview of harness components such as orchestration, memory, guardrails, feedback loops, runtime observability, and safe scale-out.

• Building the AI-Native Platform: What Engineers Need to Scale Successfully Explains how platform teams shift from tool stitching to AI-native operating models.

• Platform Engineering Becomes the Control Plane for Enterprise AI Argues that platform teams should expose governed workflows, RBAC, validation, and agent-specific authorization instead of giving agents raw human privileges.

• From Cloud-Native to AI-Native: Why Platform Engineering Is Becoming the Enterprise OS Shows why platform control, cost governance, and reliability become more critical with AI workloads.

• AI in Platform Engineering: Promise, Plateau and the Path Forward Discusses practical limits, risks, and control points for steady AI adoption in platform teams.

• Roadblocks to an AI-Native Future: What’s Holding Platform Teams Back Identifies constraints around policy design, cost, and quality in AI-native operations.

• I’ve Just Been Made a Platform Engineer: Now What? Provides pragmatic onboarding guidance for platform engineers moving to AI-native modernization.

Harness Design Primitives

• Run Long-Horizon Tasks with Codex Shows how planning artifacts such as Plan.md and Implement.md become reusable harness state for long-running work.

• The next evolution of the Agents SDK OpenAI's April 2026 update to the Agents SDK: model-native harness, native sandbox execution, configurable memory, filesystem tools, and built-in support for MCP, skills, and AGENTS.md patterns.

• Scaling Managed Agents: Decoupling the brain from the hands Anthropic's architecture note on separating session, harness, and sandbox so long-horizon agents can recover from failures and run across changing infrastructure.

• ReAct: Synergizing Reasoning and Acting in Language Models The canonical reasoning-action-observation loop paper behind most modern agent harness designs.

• Plan-and-Execute Agents Separates planner and executor layers so long-horizon tasks can be decomposed and resumed more reliably.

• Effective Harnesses for Long-Running Agents Details structured handoff patterns, persistent progress tracking, and session reuse for tasks that exceed one context window.

• Effective Context Engineering for AI Agents Reframes prompt design as a broader context-shaping problem across system prompts, tools, history, and state.

• Harness engineering and agent feedback: Exploring AI coding sensors Adds feedback sensors to feed-forward skills and guardrails, emphasizing deterministic signals such as tests, linters, and runtime checks.

• Prompt Caching — Claude API Docs Documents one of the most practical harness-level optimizations for reducing agent cost and latency.

• LangGraph — Low Level Concepts Models loop control explicitly with typed state, graph edges, checkpoints, and resumable execution.

• Closing the Knowledge Gap with Agent Skills Explains how reusable skills act as durable harness components for reducing context and capability gaps.

• Model Context Protocol Introduction Defines the interoperability layer for exposing external tools and services to agents in a consistent way.

• How Middleware Lets You Customize Your Agent Harness Shows how policy enforcement, retries, fallbacks, and dynamic tool injection can live in harness middleware instead of prompt text.

• LangGraph — Human-in-the-Loop Concepts Covers interrupts, breakpoints, approvals, and resumption patterns for human-gated agent workflows.

AI Native Engineering

• The AI-Native Software Development Lifecycle: A Theoretical and Practical New Methodology Defines AI-native SDLC design where planning, coding, testing, and release are AI-participatory by default.

• Towards the Next Generation of Software: Survey on AI-Native Applications Summarizes architecture and quality constraints seen in large AI-native software systems.

• AI-NativeBench: A White-Box Agentic Benchmark for AI-Native Systems Evaluates agent behavior, control flow, and engineering governance beyond model-level metrics.

• Vibe Coding: Toward a Semantic and Intent-Driven AI-Native Paradigm Examines intent-first AI coding and its impact on maintainability and testing approaches.

• Compiler.next: A Search-Based Compiler to Power the AI-Native Future of Software Engineering Explores compiler-level composition for scale, traceability, and quality control in AI-assisted coding.

• SemaClaw: A Step Towards General-Purpose Personal AI Agents through Harness Engineering Presents a personal-agent framework with DAG orchestration, PermissionBridge safety, three-tier context management, and agentic wiki skills.

• Meta-Engineering Harnesses for AI-Native Software Production: A Contract-Driven Adversarial Verification Architecture with Early Deployment Report Introduces contract compilation, role-specialized agents, independent and adversarial verification, failure arbitration, and outer-loop calibration for ongoing software production.

• I Saw the Future of Platform Engineering — and It’s Called AI Native Dev Explains how platform operations and handoff patterns change when teams adopt AI-native workflows.

2) Chinese & WeChat Articles

• Gartner 2024 Strategic Technology Trends: Platform Engineering and AI Official-style Chinese article discussing enterprise AI transformation, governance, and organization.

• KubeCon/CloudNativeCon 2023 Platform Engineering Sessions (WeChat Collection) Collection of platform and AI engineering talks, including implementation insights.

• AI Agent and LLM Development Perspective Shares front-end and back-end engineering considerations in AI-native development.

• AI Native Software Engineering Practical notes on requirement flow, implementation, testing, and collaboration rhythm for AI-native teams.

• AI Native Is Becoming Mainstream: Performance and Cost Tradeoffs Discusses platform stability, delivery consistency, and governance in AI-native delivery.

• AI Native Software Engineering (ZhiHu) Article-level discussion of team workflow and process design for AI-native teams.

3) Reference Implementations & Open Source Tools

Tutorials & Educational

• anthropics/claude-cookbooks — 37,517 stars Anthropic's official notebooks covering orchestrator-worker, tool use, context compaction, and Agent SDK patterns.

• huggingface/smolagents — 26,463 stars A deliberately small codebase for understanding the full harness surface, including tools, memory, and sandbox integration.

• shareAI-lab/learn-claude-code — 48,944 stars Step-by-step deconstruction of Claude Code's harness, useful for learning loop design, skills, and context management.

Coding & Agent Harnesses

• OpenHands/OpenHands — 70,670 stars A production-grade coding agent architecture with runtime isolation, event streaming, and controller layers worth studying.

• browser-use/browser-use — 86,220 stars A clean browser-agent harness showing how DOM state, actions, and retries fit into a minimal loop.

• SWE-agent/SWE-agent — 18,933 stars Demonstrates task-specific agent-computer interfaces for file search, editing, and controlled software repair.

• Aider-AI/aider — 42,897 stars A strong reference for multi-file editing, planner/coder separation, and git-aware coding workflows.

• google/adk-python — 18,777 stars Google's agent framework for multi-agent orchestration, tool registration, sessions, and evaluation pipelines.

• pydantic/pydantic-ai — 16,124 stars Type-safe agent framework that turns tool contracts and structured outputs into explicit schema-level boundaries.

• modelcontextprotocol/servers — 83,065 stars The main reference implementation set for MCP servers and external capability wiring.

• microsoft/playwright-mcp — 30,349 stars A browser automation MCP server that shows how GUI/web tooling can be exposed safely to agents.

Harness / Platform Engineering

• harness/harness — 33,987 stars Unified delivery platform with CI/CD, release governance, and policy-aware automation.

• argoproj/argo-cd — 22,420 stars GitOps control plane for standardized Kubernetes deployment governance.

• backstage/backstage — 32,887 stars Internal Developer Platform core with catalogs, templates, and self-service workflows.

• hashicorp/terraform — 48,024 stars Industry-standard Infrastructure as Code for environment standardization.

• opentofu/opentofu — 28,203 stars Terraform-compatible IaC implementation with open governance orientation.

• pulumi/pulumi — 24,947 stars Multi-language infrastructure platform with policy-friendly abstractions.

• bytebase/bytebase — 13,847 stars Database change workflow with approvals and review-oriented safety controls.

• crossplane/crossplane — 11,525 stars Controller-driven cloud resource abstraction layer for platform teams.

• tektoncd/pipeline — 8,919 stars Kubernetes-native CI/CD pipeline platform for repeatable build/deploy flows.

• keptn/keptn — 1,778 stars Event-driven application lifecycle automation with policy hooks.

• kubernetes-sigs/cluster-api — 4,146 stars Declarative cluster lifecycle APIs for platform-scale operations.

• loft-sh/vcluster — 11,062 stars Virtual cluster approach for tenant isolation and environment scaling.

• open-feature/flagd — 887 stars Feature flag service for progressive delivery and risk gating.

• KusionStack/kusion — 1,284 stars Platform-oriented intent and resource orchestration for unified delivery.

• KusionStack/karpor — 1,685 stars Kubernetes observability toolkit for complex platform estates.

• seal-io/walrus — 440 stars Self-service platform building blocks with templates and policy modules.

• projectsveltos/addon-controller — 493 stars Multi-cluster add-on orchestration and policy synchronization.

• briefercloud/layerform — 1,207 stars Layered IaC platform pattern for environment and template governance.

• open-policy-agent/conftest — 3,146 stars Reusable policy testing in CI/CD for platform-level compliance gates.

• cnoe-io/ai-platform-engineering — 328 stars AI platform engineering reference implementation with workflow guardrails.

• kbst/terraform-kubestack — 707 stars Modular Terraform toolkit for Kubernetes platform stack standardization.

AI Native Engineering Tools

• openai/openai-agents-python — 20,243 stars Standardized orchestration framework for building reproducible AI agents.

• langchain-ai/langchain — 130,796 stars Core LLM orchestration framework with broad tooling and context integration.

• langchain-ai/langgraph — 27,282 stars Graph-based orchestration for durable and stateful agent workflows.

• microsoft/autogen — 56,096 stars Multi-agent framework for planning, delegation, and shared context.

• crewAIInc/crewAI — 47,012 stars Role-based agent orchestration suited for cross-functional workflow decomposition.

• run-llama/llama_index — 47,918 stars Retrieval-augmented generation and context engineering foundation project.

• ComposioHQ/composio — 27,483 stars Secure model-to-tool integration layer with strong connector coverage.

• BerriAI/litellm — 40,092 stars LLM gateway for provider normalization, routing, and cost governance.

• tensorzero/tensorzero — 11,125 stars AI operations stack for inference routing, eval, and optimization feedback.

• qdrant/qdrant — 29,802 stars High-performance vector database for semantic retrieval in AI-native applications.

• chroma-core/chroma — 26,792 stars Lightweight vector store for RAG workloads and prototype iteration.

• langfuse/langfuse — 23,622 stars LLM observability and tracing for prompts, spans, and quality metrics.

• n8n-io/n8n — 180,729 stars AI-native workflow engine for visual automation and self-hosted deployment.

• ToolJet/ToolJet — 37,642 stars Internal-tool platform for AI-native internal applications and data workflows.

• alibaba/higress — 7,891 stars Cloud-native gateway with AI gateway and traffic management features.

• open-gitagent/gitagent — 1,056 stars Standardization-focused experiment for interoperable AI agent definitions.

• phodal/auto-dev — 4,406 stars AI-native development framework covering the full software lifecycle.

• TaskingAI/TaskingAI — 5,378 stars Open-source platform for building AI-native applications and operations layers.

• OpenBMB/IoA — 812 stars A Chinese-oriented collection around AI-native architecture and AI-native workflow experiments.

• Voltagent/voltagent — 6,935 stars Reliable AI agent execution framework for iterative production workflows.

• AGI-Eval-Official/CATArena — 62 stars Agentic benchmark dataset and evaluation tooling for engineering tasks.

• ServiceNow/AgentLab — 541 stars Evaluation-oriented platform for testing agent workflows in realistic enterprise tasks.

Evaluation Frameworks & Agent Benchmarks

• openai/evals — 18,536 stars OpenAI's framework and registry for evaluating LLMs and LLM systems, including private workflow-specific evals.

• UKGovernmentBEIS/inspect_ai — 2,127 stars UK AI Security Institute's eval framework with built-in support for tool use, multi-turn dialog, prompt engineering, and model-graded scoring.

• EleutherAI/lm-evaluation-harness — 12,698 stars Widely used model evaluation harness with many benchmark task implementations, useful as a baseline layer before agent-specific evals.

• stanford-crfm/helm — 2,800 stars Stanford CRFM's holistic evaluation framework for reproducible and transparent model and multimodal evaluation.

• SWE-bench/SWE-bench — 5,015 stars Canonical repository-level issue-resolution benchmark and execution harness for coding agents.

• microsoft/SWE-bench-Live — 192 stars Continuously updated SWE-bench-style benchmark with monthly curation, MultiLang, and Windows task variants to reduce contamination and staleness.

• harbor-framework/terminal-bench — 2,264 stars Terminal agent benchmark for hard shell tasks across software engineering, machine learning, security, data science, and related workflows.

• EuniAI/TerminalWorld — 12 stars New May 2026 benchmark and data engine derived from real terminal recordings; useful for tracking authentic shell-workflow evaluation.

• sierra-research/tau-bench — 1,246 stars Tool-Agent-User benchmark for multi-turn enterprise customer-service workflows with APIs and policy constraints.

• sierra-research/tau2-bench — 1,237 stars Current tau-bench family repo, now including knowledge, voice, task-quality fixes, and richer tool-user simulation modes.

• ServiceNow/BrowserGym — 1,229 stars Gym-style environment for web agents, bundling MiniWoB, WebArena, VisualWebArena, WorkArena, AssistantBench, OpenApps, and TimeWarp.

• xlang-ai/OSWorld — 2,875 stars Multimodal computer-use benchmark in real OS environments, useful for desktop-agent harness and GUI automation evaluation.

• claw-bench/claw-bench — 171 stars New agent benchmark with pytest-based verifiers and broad task domains; useful to watch as a fresh harness-oriented benchmark experiment.

Sandboxing, Observability & Evaluation

• e2b-dev/E2B — 11,594 stars MicroVM sandboxes for agent tool loops, useful when code execution must be isolated from the host environment.

• daytonaio/daytona — 71,416 stars Persistent workspace and sandbox infrastructure for longer-running or stateful agent tasks.

• promptfoo/promptfoo — 19,587 stars Practical LLM and agent regression testing with YAML configs, assertions, and CI integration.

• confident-ai/deepeval — 14,534 stars Open-source evaluation framework with built-in metrics for agent quality, RAG correctness, and tool behavior.

• traceloop/openllmetry — 6,981 stars OpenTelemetry-based instrumentation for tracing LLM calls, agent steps, and tool execution.

• Arize-ai/phoenix — 9,182 stars Self-hostable tracing and evaluation UI for inspecting and replaying agent workflows.

4) Security, Evals & Related Lists

Security & Permissions

• Beyond Permission Prompts Argues for structured authorization systems instead of relying on free-form natural-language permission prompts.

• Trustworthy agents in practice Explains agent risk through the four-layer model of model, harness, tools, and environment, then maps trustworthy-agent principles to concrete product controls.

• How we contain Claude across products Anthropic's containment lessons across Claude.ai, Claude Code, and Claude Cowork, emphasizing environment boundaries, sandbox and VM patterns, egress controls, and tool-output inspection.

• Designing AI agents to resist prompt injection Reframes prompt injection as a social-engineering problem and argues for source-sink controls, confirmations, and sandboxed action boundaries instead of input filtering alone.

• Model Context Protocol — Authorization The MCP authorization spec for OAuth-style external service access in agent environments.

• Claude Code auto mode: a safer way to skip permissions Details a dual-layer defense using prompt-injection probes and action classifiers to reduce approval fatigue without falling back to fully unsafe no-permission mode.

• Prompt Injection — Simon Willison's Series A practical, field-tested explanation of indirect prompt injection risk in tool-using agents.

• OWASP LLM01:2025 — Prompt Injection A concise security taxonomy for defining prompt-injection threat models and review checklists.

• tldrsec/prompt-injection-defenses — 670 stars A defense catalog covering input filtering, output sanitization, isolation, and trust-boundary hardening.

Evals & Verification

• Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies May 2026 benchmark for workspace-level agents, stressing large file dependency graphs, cross-file retrieval, contextual reasoning, and adaptive decision-making.

• TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks May 2026 benchmark generated from in-the-wild terminal recordings, complementing expert-curated terminal benchmarks with authentic shell workflows.

• SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering? Evaluates whether injected agent skills materially improve real software engineering tasks, with paired runs and execution-based verification.

• Demystifying Evals for AI Agents Explains how to design outcome, process, and trajectory evaluations that match real agent behavior.

• Testing Agent Skills Systematically with Evals OpenAI's practical guide to regression testing for skills, trace checks, and layered deterministic validation.

• Agent Evaluation Readiness Checklist A useful checklist for deciding whether an agent system is ready for stable CI and release gating.

• Evaluating Skills Shows how curated skills materially change agent outcomes and how to benchmark those gains.

• Quantifying Infrastructure Noise in Agentic Coding Evals Demonstrates that infrastructure configuration alone can swing coding-agent benchmark results materially.

• Towards a Science of AI Agent Reliability Proposes reliability metrics that separate capability gains from true operational robustness.

• VeRO: An Evaluation Harness for Agents to Optimize Agents Introduces a dedicated evaluation harness for agent-optimizing-agent workflows with versioned snapshots, budget-controlled runs, and structured traces.

Related Awesome Lists

• Meirtz/Awesome-Context-Engineering — 3,030 stars A broader survey focused on context engineering, RAG, prompt shaping, and context-window design.

• hesreallyhim/awesome-claude-code — 36,874 stars A focused collection of Claude Code resources, workflows, tools, and ecosystem projects.

• appcypher/awesome-mcp-servers — 5,370 stars A useful companion list for discovering MCP servers that expose external tools and services to agents.

• e2b-dev/awesome-ai-agents — 27,086 stars A wider landscape survey of agent projects and frameworks beyond harness-specific concerns.