Usage

May 8, 2026 · View on GitHub

Requires Go 1.26.1+ (aligned with go.mod and security fixes). For a first-time setup and full core-feature walkthrough (快速体验 vs 完整运行时), see Get Started.

Starting services

API

# Recommended: merged config (api + model)
go run ./cmd/api

# Default listen :8080; override via api.port in configs/api.yaml

Startup loads configs/api.yaml and configs/model.yaml. If model.defaults.llm and model.defaults.embedding are set, the full query_pipeline and ingest_pipeline (retrieve + generate, parse + split + embed + index) are registered automatically.

Worker (offline tasks)

go run ./cmd/worker

Uses configs/worker.yaml; consumes from the task queue and runs eino ingest_pipeline (queue is currently a placeholder implementation).

CLI

go run ./cmd/cli

For debugging and admin; subcommands and usage are in CLI (cli.md).

Environment variables and configuration

API Key: In configs/model.yaml use api_key: "${OPENAI_API_KEY}"; it is substituted at runtime.
Planner (v1 Agent):
- PLANNER_TYPE=rule: Uses RulePlanner - returns a fixed single-node llm TaskGraph. No LLM needed for planning, useful for Executor/DAG debugging. Startup logs show "Worker 使用规则规划器".
- PLANNER_TYPE=llm (default): Uses LLMPlanner - generates TaskGraph dynamically using LLM. Requires LLM configuration in model.yaml. Startup logs show which planner is active.
- Note: Even with RulePlanner, executing llm nodes still requires LLM configuration.
Secrets: Do not commit real API keys; use environment variables or a secrets manager.
Storage: API defaults to memory storage (data lost on restart); for production configure MySQL/Milvus etc. (implement the corresponding Store). The vector store is configurable via storage.vector (type, collection); the default collection name is used for both ingest and query. Optional storage.ingest (batch_size, concurrency) tunes the ingest pipeline. See config.md for full storage and ingest options.
Ingest logging: The ingest pipeline logs each step (loader, parser, splitter, embedding, indexer) with ingest_id, ingest_step, doc_id, chunks, and duration_ms for observability.
Tracing: Enable OpenTelemetry under monitoring.tracing in configs/api.yaml; when export_endpoint is unset, the OTEL_EXPORTER_OTLP_ENDPOINT env var is used. See Tracing (tracing.md).
For per-file config reference see Configuration (config.md).

Typical flows

1. Upload a document

curl -X POST http://localhost:8080/api/documents/upload \
  -F "file=@/path/to/your.pdf"

On success, ingest_pipeline runs: load → parse → split → embed → write to default vector index and metadata.

2. List documents

curl http://localhost:8080/api/documents/

3. Runtime-first API (recommended)

# Create run
curl -X POST http://localhost:8080/api/runs \
  -H "Content-Type: application/json" \
  -d '{
    "workflow_id": "agent_message",
    "input": {"goal":"Your question"},
    "idempotency_key": "demo-run-001"
  }'

# Poll run status
curl http://localhost:8080/api/runs/<run_id>

# Inspect run events
curl http://localhost:8080/api/runs/<run_id>/events

Use runtime-first APIs for new integrations. They are the canonical control surface for submission and state transitions.

4. Agent facade (legacy compatibility)

# Create Agent
curl -X POST http://localhost:8080/api/agents \
  -H "Content-Type: application/json" \
  -d '{"name": "my-agent"}'
# Returns {"id": "agent-xxx", "name": "my-agent"}

# Send message (triggers planning and execution)
curl -X POST http://localhost:8080/api/agents/<agent-id>/message \
  -H "Content-Type: application/json" \
  -d '{"message": "Your question"}'
# Returns 202 Accepted with job_id, e.g. {"status":"accepted","agent_id":"...","job_id":"job-xxx"}

# Poll job status
curl http://localhost:8080/api/agents/<agent-id>/jobs/<job_id>
# Returns job details: id, agent_id, goal, status (pending|running|completed|failed), cursor, retry_count, created_at, updated_at

# List jobs for this Agent (optional query: status, limit)
curl "http://localhost:8080/api/agents/<agent-id>/jobs?limit=20&status=completed"

# Agent state
curl http://localhost:8080/api/agents/<agent-id>/state

# List all agents
curl http://localhost:8080/api/agents

v0.8 execution path: Message is written to Session → dual-write creates Job (if JobEventStore is configured: append JobCreated to event stream, then state JobStore.Create) → Scheduler pulls Pending jobs from state JobStore → Runner.RunForJob (Steppable + node-level Checkpoint) → PlanGoal produces TaskGraph → compile to eino DAG → execute node by node → update Job status on completion/failure. RAG can be used via workflow nodes chosen by the planner.

Job storage (event stream): The event stream interface (ListEvents, Append, Claim, Heartbeat, Watch) supports crash recovery, multiple workers, and audit replay; the API currently uses an in-process memory implementation.

Scheduler: Runs in the API process, pulls jobs and executes them. Scheduler params (MaxConcurrency, RetryMax, Backoff) are set in app code (e.g. concurrency 2, retry 2, backoff 1s); see internal/app/api/app.go.

Control / Data plane (Postgres): When jobstore.type=postgres, the API is control-plane only (create job, query, cancel); it does not start the Scheduler or run any jobs. All execution is done by Workers via Postgres Claim. API restart or scale does not affect already-claimed jobs. See design/services.md §7.

5. Query (deprecated; prefer runtime-first or Agent facade)

curl -X POST http://localhost:8080/api/query \
  -H "Content-Type: application/json" \
  -d '{"query": "Your question", "top_k": 10}'

Uses query_pipeline: embed query → retrieve → LLM generate answer. Deprecated; use runtime-first submission (POST /api/runs) or legacy facade POST /api/agents/{id}/message.

6. Batch query

curl -X POST http://localhost:8080/api/query/batch \
  -H "Content-Type: application/json" \
  -d '{"queries": [{"query": "Question 1"}, {"query": "Question 2"}]}'

API endpoint summary

Method	Path	Description
GET	/api/health	Health check
v1 Agent (legacy facade)
POST	/api/agents	Create agent
GET	/api/agents	List all agents
POST	/api/agents/:id/message	[legacy facade] Send message (creates job, 202 + job_id); optional `Idempotency-Key` header
GET	/api/agents/:id/state	Agent state (status, current_task, last_checkpoint)
GET	/api/agents/:id/jobs	[legacy facade] List jobs for this agent (?status=, ?limit=)
GET	/api/agents/:id/jobs/:job_id	[legacy facade] Single job (poll status)
Runs
POST	/api/runs	Create a new run
GET	/api/runs/:id	Get run details
GET	/api/runs/:id/events	Get run events
POST	/api/runs/:id/tool-calls	Upsert tool call result
POST	/api/runs/:id/pause	Pause a run
POST	/api/runs/:id/resume	Resume a run
POST	/api/runs/:id/human-decisions	Inject human decision
Agent legacy
POST	/api/agent/run	[legacy] Run agent
POST	/api/agent/resume	[legacy] Resume agent checkpoint
POST	/api/agent/stream	Run agent with streaming
Execution trace
GET	/api/jobs/:id/events	Raw event stream (id, type, payload, created_at)
GET	/api/jobs/:id/trace	Timeline and execution_tree, node timings
GET	/api/jobs/:id/trace/page	Same as trace, HTML page
GET	/api/jobs/:id/replay	Read-only replay
POST	/api/agents/:id/resume	Resume execution
POST	/api/agents/:id/stop	Stop execution
Documents and knowledge
POST	/api/documents/upload	Upload document
GET	/api/documents/	List documents
GET	/api/documents/:id	Document details
DELETE	/api/documents/:id	Delete document
GET	/api/knowledge/collections	List collections
POST	/api/knowledge/collections	Create collection
DELETE	/api/knowledge/collections/:id	Delete collection
Documents (async)
POST	/api/documents/upload/async	Upload document asynchronously
GET	/api/documents/upload/status/:task_id	Get upload task status
Query (deprecated)
POST	/api/query	Single query (prefer runtime-first; Agent facade for compatibility)
POST	/api/query/batch	Batch query
Legacy agent
POST	/api/agent/run	Run agent by session (query + session_id)
System
GET	/api/system/status	System status (workflows, agents)
GET	/api/system/metrics	Metrics
GET	/api/system/workers	Worker status and info
Observability
GET	/api/observability/summary	Observability summary
GET	/api/observability/stuck	Stuck jobs list
GET	/api/trace/overview/page	Trace overview HTML page
RBAC
GET	/api/rbac/role	Get current user role
POST	/api/rbac/role	Assign role to user
POST	/api/rbac/check	Check user permissions

Document, knowledge, agent, and query routes may have auth middleware; see internal/api/http/router.go.

Execution trace (explainable execution)

After sending a message you get a job_id. Use these endpoints to see what the job did:

GET /api/jobs/:id/events: Full event stream (job_created, plan_generated, node_started, node_finished, command_emitted, command_committed, tool_called, tool_returned, job_completed, etc.) to reconstruct the User → Plan → nodes → tool calls chain.
GET /api/jobs/:id/trace: Timeline, node timings, and execution_tree for an explainable view.
GET /api/jobs/:id/trace/page: Same as trace, as an HTML page.

Event semantics and tree derivation are in design/execution-trace.md.

FAQ

Job and event stream: The returned job_id is written to both the event stream (JobCreated) and the state JobStore for future replay or multi-worker consumption; execution is still driven by the state JobStore + Scheduler.
Idempotency-Key: POST /api/agents/:id/message supports header Idempotency-Key. Duplicate requests with the same key (e.g. retries) return the existing job_id (202) and do not create a new job or rewrite Session/Plan.
Migration field: legacy POST /api/agents/:id/message responses include optional runtime_submission object to expose canonical runtime mapping (canonical_api, run_id, run_status).
Poison jobs: When a job keeps failing, after max_attempts (Scheduler retry_max, Worker max_attempts) it is marked Failed and no longer scheduled; see design/poison-job.md.
v1 Agent vs /api/query: v1 Agent uses Agent + Session + plan → TaskGraph → eino DAG as the only path; RAG is an optional tool. /api/query still hits query_pipeline directly and is deprecated; use Agent messages for new usage.
PLANNER_TYPE=rule: Uses RulePlanner for debugging; returns fixed TaskGraph without LLM. Worker logs show "Worker 使用规则规划器". Use when LLMPlanner generates invalid graphs or for stable testing.
No OPENAI_API_KEY: API still starts but will not register real LLM/Embedding query and ingest workflows; query/upload may use placeholders or fail. With RulePlanner, planning does not need LLM, but executing llm nodes still requires LLM config.
Memory storage: Default metadata and vector are in-memory; data is lost on restart. Configure and implement a persistent store for production.
Config not applied: Ensure the API uses LoadAPIConfigWithModel (cmd/api does) and that configs/model.yaml has defaults.llm, defaults.embedding and the matching provider/model keys.
Tracing: Set monitoring.tracing.enable: true and export_endpoint in configs/api.yaml (or OTEL_EXPORTER_OTLP_ENDPOINT); otherwise no traces are sent. Use Jaeger or another OTLP backend; see tracing.md.
Authentication required: In production mode (AETHERIS_ENV=production), JWT authentication is required. Set auth.jwt.secret in configs/api.yaml and include Authorization: Bearer <token> in requests. See security.md.
LLMPlanner fails: If you see "TaskGraph 存在环，无法拓扑排序" in Worker logs, the LLM is generating an invalid graph with cycles. Set PLANNER_TYPE=rule to use RulePlanner for debugging. Ensure LLM model is capable of generating valid JSON plans.
Worker not processing jobs: Ensure Worker is running and connected to the same PostgreSQL database as the API. Check Worker logs for connection errors. With jobstore.type=postgres, jobs must be claimed by a Worker.
Job stuck in pending: If using Postgres jobstore, at least one Worker must be running to claim and process jobs. Without Workers, jobs remain pending indefinitely.
Multiple Workers same job: Aetheris uses lease fencing to prevent duplicate execution. Each job has a lease token that expires; only the Worker with the valid lease can execute. See design/scheduler-correctness.md.
Resume after crash: Aetheris automatically resumes jobs from the last checkpoint after Worker or API crash. The event stream provides full replay capability.
Human-in-the-loop: Use POST /api/jobs/:id/stop to pause, then inject human decision via POST /api/runs/:id/human-decisions before resuming.
Evidence graph (M3): Access via GET /api/jobs/:id/evidence-graph to get the complete decision provenance graph. Requires M3 features enabled.
Forensics export: Use POST /api/forensics/query for advanced queries or POST /api/jobs/:id/export to export complete job data for auditing.

Architecture and module roles are in design/; deployment steps are in each deployments/ README.