Connect to Remote Agents

August 21, 2026 · View on GitHub

Use any agent, anywhere, as if it were local. Real-time UI updates included.


Architecture

┌────────────────────────────────────────────────────────────────────────────┐
│                            YOUR APPLICATION                                 │
│  ┌──────────────┐        ┌──────────────┐        ┌──────────────────────┐  │
│  │ React/Vue    │        │  Python      │        │  Swift/Kotlin        │  │
│  │ useAgentForHuman()   │        │  connect()   │        │  connect()           │  │
│  └──────┬───────┘        └──────┬───────┘        └──────────┬───────────┘  │
│         │                       │                           │              │
└─────────┼───────────────────────┼───────────────────────────┼──────────────┘
          │                       │                           │
          └───────────────────────┼───────────────────────────┘


                    ┌─────────────────────────────┐
                    │  WebSocket /ws/input        │
                    │  wss://oo.openonion.ai      │
                    └──────────────┬──────────────┘


          ┌────────────────────────────────────────────────────┐
          │                   RELAY SERVER                      │
          │  ┌──────────────┐  ┌──────────────┐  ┌───────────┐ │
          │  │ /ws/announce │  │ /ws/input    │  │ /ws/lookup│ │
          │  │ Agents       │  │ Clients      │  │ Discovery │ │
          │  └──────┬───────┘  └──────┬───────┘  └───────────┘ │
          │         │                 │                         │
          │         │  active_connections {address → WebSocket} │
          │         │  pending_outputs {input_id → Future}      │
          │         │                                           │
          └─────────┼─────────────────┼─────────────────────────┘
                    │                 │
                    │     ┌───────────┘
                    │     │
                    ▼     ▼
          ┌────────────────────────────────────────────────────┐
          │                    AGENT                            │
          │  host(agent) → /ws/announce → ANNOUNCE → ready     │
          │                                                     │
          │  INPUT received → agent.input(prompt) → OUTPUT      │
          └────────────────────────────────────────────────────┘

Lifecycle

1. Agent Registers (Server Side)

from connectonion import Agent, host

agent = Agent("my-agent", tools=[...])
host(agent)  # Connects to /ws/announce, sends ANNOUNCE

The agent:

  1. Connects WebSocket to wss://oo.openonion.ai/ws/announce
  2. Sends ANNOUNCE: {type, address, summary, endpoints, signature}
  3. Relay stores in active_connections[address] = websocket
  4. Agent waits for INPUT messages

2. Client Connects (Any Platform)

from connectonion import connect

agent = connect("0x123abc...")
response = agent.input("Hello")

The client:

  1. connect(address) creates RemoteAgent instance
  2. input(prompt) opens WebSocket to /ws/input
  3. Sends INPUT: {type: "INPUT", input_id, to: "0x...", prompt, session?}
  4. Relay looks up active_connections[to]
  5. Relay forwards INPUT to agent's WebSocket

3. Agent Processes

Relay → tagged messages → Agent serve_loop

                            └─ routes by session_id → run_ws_session()
                               (same protocol handler as direct connections)

                               ├─ agent.input(prompt)

                               ├─ Streaming events:
                               │  ← tool_call, tool_result, thinking, ask_user

                               └─ OUTPUT → relay WS → Relay → Client

The agent calls run_ws_session() directly with relay transport adapters — no loopback WebSocket. Relay connections get the same features as direct connections (streaming, tool calls, ask_user, session recovery) because both paths use the same protocol handler.

4. Client Receives Response

response = agent.input("Hello")

response.text   # "Hello! How can I help?"
response.done   # True (complete) or False (needs more input)

agent.ui        # All events for rendering
agent.status    # 'idle' | 'working' | 'waiting'

Direct tool execution (remote.call)

input() hands the remote LLM a task and waits for it to reason. When you already know the exact tool and arguments, remote.call() runs it directly — no LLM, no session — and returns the raw output (text or a base64 screenshot):

remote = connect("0x3d40...")
print(remote.call("bash", command="co status").text)
shot = remote.call("bash", command="co browser take_screenshot")   # .images extracts base64

Gated by the host's .co/host.yaml whitelist. From the shell it's `co call

`. Full reference: [remote-call.md](remote-call.md).

Connection Modes

Via Relay (Default)

Uses agent address to route through relay server:

# Python
agent = connect("0x3d4017c3...")
// TypeScript
const agent = connect("0x3d4017c3...");

Direct to Deployed Agent

For agents deployed via co deploy, connect directly to their URL:

// TypeScript - bypass relay
const agent = connect("agent-name", {
  directUrl: "https://my-agent.agents.openonion.ai"
});

The relay stores agent-provided endpoints and can return them for direct connections. The SDKs do not automatically probe endpoints yet; they use relay by default (Python) or directUrl when provided (TypeScript).

To implement smarter routing:

  1. Lookup endpoints for the agent via relay:
    • WebSocket /ws/lookup with GET_AGENT
    • HTTP /api/agents/{address}
  2. Try direct endpoints first (if any):
    • Prefer ws:///http:// endpoints that are reachable from your network.
    • If you are on the same LAN, a private IP (RFC1918) endpoint may be fastest.
  3. Fallback to relay /ws/input if direct endpoints fail.

The relay does not determine whether an endpoint is “local” or “public”; it simply returns what the agent announced. There is no WebRTC support in the relay server today. TODO: Add WebRTC-style ICE candidates (host/srflx/relay) and connectivity checks so clients can automatically pick the best direct path.

Lookup via WebSocket

// Client → /ws/lookup
{ "type": "GET_AGENT", "address": "0x3d4017c3..." }
// Server → client
{
  "type": "AGENT_INFO",
  "agent": {
    "address": "0x3d4017c3...",
    "summary": "translator agent",
    "endpoints": ["ws://192.168.1.10:8000/ws"],
    "last_announce": "2024-01-15T10:23:45Z",
    "online": true
  }
}

Lookup via HTTP

curl https://oo.openonion.ai/api/agents/0x3d4017c3...
{
  "online": true,
  "endpoints": ["ws://192.168.1.10:8000/ws"],
  "last_seen": "2024-01-15T10:23:45Z"
}

Connection Reliability & Recovery

The ConnectOnion client (TypeScript/Python) automatically handles connection failures and recovers results seamlessly.

Automatic Keep-Alive

Server sends PING every 30 seconds:

  • Client automatically responds with PONG
  • Keeps connection alive through proxies and firewalls
  • Detects dead connections within 60 seconds

No configuration needed - handled automatically by the SDK.

Extended Timeout

Default timeout: 10 minutes (600 seconds)

Long-running agent tasks have plenty of time to complete:

// TypeScript - default 10 minutes
const response = await agent.input("Analyze this large dataset");

// Override if needed (5 minutes)
const response = await agent.input("Quick task", 300000);

Automatic Session Recovery

If the WebSocket connection fails (network drop, timeout, page refresh), the SDK automatically polls the server to retrieve your result:

1. Connection fails or times out

2. SDK polls GET /sessions/{session_id} every 10s

3. Server returns result when ready

4. SDK returns result to your code

5. You get the result as if nothing happened! ✅

What this means for you:

  • ✅ Page refresh during long tasks? No problem.
  • ✅ Network hiccup? Result still delivered.
  • ✅ Connection timeout? Automatically recovered.
  • ✅ Agent takes 15 minutes? You still get the result.

Configuration (TypeScript):

const agent = connect("0x123...", {
  enablePolling: true,        // Default: true
  pollIntervalMs: 10000,      // Poll every 10s (default)
  maxPollAttempts: 30         // Try for 5 minutes (default)
});

Session persistence:

  • Results stored server-side for 24 hours
  • Session ID automatically generated and tracked
  • localStorage used (browser) to survive page refreshes

Connection Lifecycle

┌─────────────────────────────────────────────────────┐
│  Normal Operation (WebSocket)                       │
├─────────────────────────────────────────────────────┤
│  1. Open WebSocket                                  │
│  2. Send CONNECT { session_id?, auth }              │
│  3. Receive CONNECTED { session_id, status }        │
│  4. Send INPUT { prompt }                           │
│  5. Receive PING every 30s, respond with PONG       │
│  6. Receive streaming events                        │
│  7. Receive OUTPUT (result)                         │
│  8. Keep WS open for next message                   │
└─────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────┐
│  Recovery Mode (HTTP Polling)                       │
├─────────────────────────────────────────────────────┤
│  1. WebSocket fails/timeout                         │
│  2. SDK polls: GET /sessions/{session_id}           │
│  3. Server responds: {"status": "running"}          │
│  4. Wait 10s, poll again                            │
│  5. Server responds: {"status": "done", "result"}   │
│  6. SDK returns result to your code                 │
└─────────────────────────────────────────────────────┘

Error Scenarios Handled

ScenarioWhat Happens
Network disconnectAutomatic polling recovers result
Page refreshSession ID in localStorage, poll for result
10-minute timeoutPolling activates, waits for completion
Server restartPolling continues, result available when server back
Connection drops mid-streamPolling recovers final result

Best Practices

1. For long-running tasks:

// Just call input() - recovery is automatic
const response = await agent.input("Process 1GB of data");
// Works even if it takes 20 minutes

2. For user feedback:

agent.on('reconnecting', () => {
  showMessage('Connection lost, recovering...');
});

agent.on('polling', () => {
  showMessage('Checking for results...');
});

3. For critical operations:

try {
  const response = await agent.input("Critical task");
  console.log("Success:", response.text);
} catch (error) {
  // Only fails if:
  // - Server down for 24+ hours
  // - Session expired (24h TTL)
  console.error("Failed:", error);
}

Message Protocol

INPUT (Client → Relay → Agent)

{
  "type": "INPUT",
  "input_id": "uuid-1234",
  "to": "0x3d4017c3...",
  "prompt": "Book a flight to Tokyo",
  "session": { "messages": [...] }
}

OUTPUT (Agent → Relay → Client)

{
  "type": "OUTPUT",
  "input_id": "uuid-1234",
  "result": "Booked! Confirmation #ABC123",
  "session": { "messages": [...updated...] }
}

Streaming Events (Agent → Client)

EventPurpose
tool_callTool execution started {id, name, args, status: "running"}
tool_resultTool completed {id, result, status: "done"}
thinkingAgent is processing
ask_userAgent needs input {text, options, multi_select}done: false

Note: Relay /ws/input does not forward streaming events. Use direct host /ws for real-time events.


FilePurpose
connectonion/network/connect.pyPython client - RemoteAgent class
connectonion/network/relay.pyAgent-side relay connection
connectonion-react/src/connect/Browser TypeScript connection and protocol implementation
connectonion-react/src/use-agent-for-human.tsuseAgentForHuman React hook
oo-api/relay/routes.pyRelay server endpoints

Quick Start

from connectonion import connect

agent = connect("0x...")

response = agent.input("Book a flight to Tokyo")
print(response.text)   # "Which date do you prefer?"
print(response.done)   # False - agent asked a question

response = agent.input("March 15")
print(response.text)   # "Booked! Confirmation #ABC123"
print(response.done)   # True

Response

response = agent.input("task")

response.text   # Agent's response or question
response.done   # True = complete, False = needs more input

Session State

current_session is synced from the server when the server includes it (direct host /ws). Relay /ws/input currently returns only OUTPUT without session data.

agent.current_session   # Synced from server when available (read-only)
agent.ui                # Client-side UI event list (input + streamed events)
agent.status            # 'idle' | 'working' | 'waiting'

UI Rendering

agent.ui contains all events for rendering. One type = one component. Streaming events are delivered only for direct host /ws connections; relay returns only OUTPUT.

agent.ui = [
    {"id": "1", "type": "user", "content": "Book a flight"},
    {"id": "2", "type": "thinking"},
    {"id": "3", "type": "tool_call", "name": "search_flights", "status": "running"},
    # ↑ When tool_result arrives, client updates this item to status: "done"
    {"id": "4", "type": "agent", "content": "Found 3 flights..."},
    {"id": "5", "type": "ask_user", "text": "Which date?", "options": ["Mar 15", "Mar 16"]},
]

Event Types

TypeComponentFields
userUser chat bubblecontent
agentAgent chat bubblecontent
thinkingLoading spinner-
tool_callTool cardname, status, result?
ask_userQuestion formtext, options, multi_select

Server → Client Mapping

Server sends two events, client merges into one UI item:

Server: tool_call   {id: "3", name: "search"}     → UI: {id: "3", status: "running"}
Server: tool_result {id: "3", result: "..."}      → UI: {id: "3", status: "done", result: "..."}

Cross-Platform SDKs

Python

from connectonion import connect

agent = connect("0x...")
response = agent.input("Book a flight")
print(response.text)   # "Which date?"
print(response.done)   # False
print(agent.ui)        # All events for rendering

TypeScript

import { connect } from 'connectonion';

const agent = connect('0x...');
const response = await agent.input('Book a flight');
console.log(response.text);   // "Which date?"
console.log(response.done);   // false
console.log(agent.ui);        // All events for rendering

Swift

import ConnectOnion

let agent = connect("0x...")
let response = try await agent.input("Book a flight")
print(response.text)   // "Which date?"
print(response.done)   // false
print(agent.ui)        // All events for rendering

Kotlin

import com.connectonion.connect

val agent = connect("0x...")
val response = agent.input("Book a flight")
println(response.text)   // "Which date?"
println(response.done)   // false
println(agent.ui)        // All events for rendering

React: useAgentForHuman() Hook

The @connectonion/react package (repo: openonion/connectonion-react) exports the React hooks and the browser connection layer with state management and localStorage persistence. React is its only peer dependency:

npm install @connectonion/react

These hooks used to ship inside the SDK at connectonion/react. That subpath was removed in connectonion@0.3.0 — import from @connectonion/react instead. Same hooks, same signatures, same localStorage keys.

Basic Usage

import { useAgentForHuman } from '@connectonion/react'

function ChatPage() {
  const {
    ui,              // ChatItem[] — all streaming events
    status,          // 'idle' | 'working' | 'waiting'
    isProcessing,    // true while agent is working
    mode,            // 'read-only' | 'auto' | 'full-access'
    turnsLeft,       // number | null
    input,           // send a message
    respond,         // answer ask_user
    respondToApproval,
    setSessionMode,
    reset,
  } = useAgentForHuman("0x3d4017c3e843...", {
    sessionId: "my-session-123"  // required — auto-persisted to localStorage
  })

  return (
    <div>
      {ui.map(item => {
        if (item.type === 'user') return <UserMsg key={item.id}>{item.content}</UserMsg>
        if (item.type === 'agent') return <AgentMsg key={item.id}>{item.content}</AgentMsg>
        if (item.type === 'thinking') return <Thinking key={item.id} />
        if (item.type === 'tool_call') return <ToolCall key={item.id} name={item.name} />
        if (item.type === 'ask_user') return (
          <AskUser
            key={item.id}
            question={item.text}
            options={item.options}
            onAnswer={(answer) => respond(answer)}
          />
        )
        if (item.type === 'approval_needed') return (
          <Approval
            key={item.id}
            tool={item.tool}
            onApprove={() => respondToApproval(true, 'once')}
            onReject={() => respondToApproval(false, 'once')}
          />
        )
        return null
      })}
      <Input onSend={(msg) => input(msg)} disabled={isProcessing} />
    </div>
  )
}

Hook Return Interface

const agent = useAgentForHuman(address, { sessionId })

// State (reactive)
agent.ui: ChatItem[]           // All events for rendering
agent.status: AgentStatus      // 'idle' | 'working' | 'waiting'
agent.isProcessing: boolean    // true while agent working
agent.mode: Mode // 'read-only' | 'auto' | 'full-access'
agent.turnsLeft: number | null
agent.error: Error | null
agent.sessionId: string

// Actions
agent.input(prompt, options?)       // Send message
agent.respond(answer)               // Answer ask_user
agent.respondToApproval(approved, scope, mode?, feedback?)
agent.submitOnboard(options)        // Submit invite code / payment
agent.setSessionMode(mode)          // Await Host acknowledgement
agent.setPrompt(prompt)             // Set persistent system prompt
agent.reset()                       // Clear conversation

Session Persistence

The hook automatically saves state to localStorage:

  • Key: co:agent:{address}:session:{sessionId}
  • Page refresh restores the full conversation
  • No manual save/load needed

Interactive Features

Agents can ask questions and request tool approval. Todo List progress is read-only session data, not an interactive gate:

// Ask User — agent needs information
// Event: { type: 'ask_user', text: 'Which city?', options: ['Sydney', 'Tokyo'] }
respond("Sydney")
respond(["Sydney", "Tokyo"])  // multi-select

// Tool Approval — agent wants to run a dangerous tool
// Event: { type: 'approval_needed', tool: 'shell', arguments: { cmd: 'rm -rf /tmp' } }
respondToApproval(true, 'once')      // approve once
respondToApproval(true, 'session')   // approve for session
respondToApproval(false, 'once', 'reject_explain', 'Too dangerous')


oo-chat: Open-Source Reference Client

oo-chat is an open-source Next.js chat client built on @connectonion/react. It's a complete working example.

Architecture

┌──────────────────────────────────────────────────┐
│  oo-chat (Next.js)                               │
│                                                   │
│  app/[address]/[sessionId]/page.tsx               │
│    └─ useAgentSDK()     ← elapsed time, pending   │
│         └─ useAgentForHuman()  ← @connectonion/react│
│              └─ connect()  ← WebSocket to agent   │
│                                                   │
│  <Chat />                                         │
│    ├─ <ChatMessages />  ← renders ui: ChatItem[]  │
│    ├─ <AskUser />       ← from pendingAskUser     │
│    ├─ <Approval />      ← from pendingApproval    │
│    └─ <ChatInput />     ← calls send()            │
└──────────────────────────────────────────────────┘
         │ WebSocket

┌──────────────────────────────────────────────────┐
│  Hosted Agent (Python)                            │
│  host(agent)                                      │
└──────────────────────────────────────────────────┘

File Structure

oo-chat/
├── app/[address]/[sessionId]/page.tsx   ← session page (uses useAgentSDK)
├── components/chat/
│   ├── chat.tsx                         ← main Chat component
│   ├── chat-input.tsx                   ← message input
│   ├── chat-messages.tsx                ← message list (renders ChatItem[])
│   ├── use-agent-sdk.ts                 ← wrapper hook around useAgentForHuman()
│   └── messages/
│       ├── tool-call.tsx                ← tool call card
│       └── tools/                       ← tool-specific presentation
└── package.json                         ← depends on @connectonion/react

How It Connects

// app/[address]/[sessionId]/page.tsx
import { useAgentSDK } from '@/components/chat/use-agent-sdk'

export default function ChatSession({ params }) {
  const { address, sessionId } = params

  const {
    ui,
    isLoading,
    elapsedTime,
    pendingAskUser,
    pendingApproval,
    mode,
    turnsLeft,
    availableModes,
    send,
    respondToAskUser,
    respondToApproval,
    setSessionMode,
    clear,
  } = useAgentSDK({ agentAddress: address, sessionId })

  return (
    <Chat
      ui={ui}
      isLoading={isLoading}
      elapsedTime={elapsedTime}
      onSend={(msg, images) => send(msg, images)}
      pendingAskUser={pendingAskUser}
      onAskUserResponse={respondToAskUser}
      pendingApproval={pendingApproval}
      onApprovalResponse={respondToApproval}
      mode={mode}
      turnsLeft={turnsLeft}
      availableModes={availableModes}
      onModeChange={setSessionMode}
    />
  )
}

useAgentSDK is a thin wrapper around useAgentForHuman() that adds elapsed time tracking and extracts ask_user and approval states for rendering.


API Reference

connect()

agent = connect("0x...", relay_url="ws://localhost:8000/ws/announce")
ParameterTypeDefaultDescription
addressstrrequiredAgent's address (0x...)
relay_urlstrproductionRelay server URL

RemoteAgent

class RemoteAgent:
    # Actions
    def input(self, prompt: str) -> Response
    def set_session_mode(self, mode: str, timeout: float = 30.0) -> None
    def reset(self) -> None

    # State (read-only)
    current_session: dict    # Full session data
    available_modes: list     # Host-advertised modes for this session
    ui: List[UIEvent]        # Shortcut to current_session['trace']
    status: str              # 'idle' | 'working' | 'waiting'

set_session_mode() uses one timeout budget for endpoint resolution, CONNECT, PING handling, and the owned OIP mode response. If it raises TimeoutError, the durable outcome is unknown: Host persistence may have completed even though the acknowledgement did not arrive. Reconnect and use the next CONNECTED state (available_modes and current_session["mode"]) as authority before retrying.

useAgentForHuman() (React)

const agent = useAgentForHuman(address, { sessionId })

// State (reactive)
agent.ui: ChatItem[]           // All events for rendering
agent.status: AgentStatus      // 'idle' | 'working' | 'waiting'
agent.isProcessing: boolean
agent.mode: Mode // 'read-only' | 'auto' | 'full-access'
agent.turnsLeft: number | null

// Actions
agent.input(prompt)            // Send message
agent.respond(answer)          // Answer ask_user
agent.respondToApproval(approved, scope)
await agent.setSessionMode(mode)
agent.reset()                  // Clear conversation

ChatItem Types (TypeScript)

type ChatItem =
  | { id, type: 'user', content, images? }
  | { id, type: 'agent', content, images? }
  | { id, type: 'thinking', status, model?, duration_ms? }
  | { id, type: 'tool_call', name, args?, status, result?, timing_ms? }
  | { id, type: 'ask_user', text, options, multi_select }
  | { id, type: 'approval_needed', tool, arguments, description? }
  | { id, type: 'tool_blocked', tool, reason, message, command? }
  | { id, type: 'onboard_required', methods, paymentAmount? }

Data Types (Python)

@dataclass
class Response:
    text: str       # Agent's response
    done: bool      # True = complete, False = needs input

State Machine

                    input()                 response.done=false
        IDLE ────────────────▶ WORKING ─────────────────────▶ WAITING
          ▲                       │                              │
          │                       │ response.done=true           │ input()
          │                       ▼                              │
          └───────────────────────────────────────────────────────

Common Patterns

Conversation Loop

agent = connect("0x...")

response = agent.input("book a flight")

while not response.done:
    answer = input(f"{response.text}: ")
    response = agent.input(answer)

print(f"Final: {response.text}")

Multiple Agents

researcher = connect("0xaaa...")
writer = connect("0xbbb...")

research = researcher.input("Research AI trends").text
article = writer.input(f"Write about: {research}").text

Complete Example

Terminal 1: Host an Agent

from connectonion import Agent, host

def search(query: str) -> str:
    return f"Found results for: {query}"

def book_flight(destination: str, date: str) -> str:
    return f"Booked flight to {destination} on {date}. Confirmation: ABC123"

agent = Agent("travel-assistant", tools=[search, book_flight])
host(agent)

Terminal 2: Connect and Use

from connectonion import connect

agent = connect("0x7a8f...")
response = agent.input("Book me a flight to Paris")

while not response.done:
    print(response.text)
    answer = input("> ")
    response = agent.input(answer)

print(f"Done: {response.text}")

Error Handling

from connectonion import connect, ConnectionError, TimeoutError

agent = connect("0x...")
response = agent.input("task")
# Errors raise exceptions - no try/except needed unless you want custom handling

Summary

# Python
agent = connect("0x...")
response = agent.input("task")
agent.ui      # All events for UI rendering
agent.status  # 'idle' | 'working' | 'waiting'
// TypeScript
const agent = connect("0x...")
const response = await agent.input("task")
agent.ui      // ChatItem[] for rendering
// React
const { ui, input, respond, respondToApproval } = useAgentForHuman("0x...", { sessionId })

Event types: user, agent, thinking, tool_call, ask_user, approval_needed, tool_blocked, plus observational Todo List state.

One event type = one UI component. Render ui and handle interactive events with respond() / respondToApproval(). Todo List progress is exposed separately.

Reference implementation: oo-chat — open-source Next.js chat client built on @connectonion/react.


Learn More

  • host.md - Host agents for remote access
  • io.md - IO interface for real-time communication