Autohand in Chrome

Autohand in Chrome connects your CLI coding agent to a Chrome extension, giving it the ability to navigate pages, fill forms, capture screenshots, read network traffic, and debug — all from your terminal.

How It Works

CLI (autohand)
  ├── /browser command creates a handoff token
  ├── Opens Chrome with the Autohand side panel
  └── Communicates via native messaging (JSON-RPC 2.0)
        │
        ▼
Chrome Extension (side panel)
  ├── Receives instructions from CLI
  ├── Executes browser tools on the active tab
  └── Returns results back to CLI

The CLI and extension communicate through Chrome's native messaging protocol. A generated Node.js bridge process (~/.autohand/chrome/native-host/host.js) translates between Chrome's length-prefixed framing and the CLI's line-based JSON-RPC.

Quick Start

1. Install the Extension

Install the Autohand Chrome extension from the Chrome Web Store or load it unpacked from your local build.

2. Connect from the CLI

# Start autohand
autohand

# In the REPL, run:
/browser

Select Open in Chrome from the menu. This will:

• Install the native messaging host (if not already installed)
• Create a handoff token for the current session
• Open Chrome with the Autohand side panel

3. Use the Side Panel

Press Cmd+E (macOS) or Ctrl+E (Windows/Linux) to toggle the side panel. The extension will automatically attach to your CLI session.

CLI Flags

autohand --browser          # Start with browser bridge enabled
autohand --no-browser       # Start with browser bridge disabled

Slash Commands

Command	Description
/browser	Open the browser integration panel with connection status
/browser disconnect	Close the browser bridge and disable it

/browser Panel

When you run /browser, you see a panel with:

• Connection: Connected (green), Disconnected (yellow), or Not installed (red)
• Status: Whether the native host is installed
• Extension: Whether the extension profile was detected

Options

Option	Description
Open in Chrome	Create a handoff and launch Chrome
Manage permissions	Open extension settings
Reconnect extension	Reinstall the native messaging host
Enabled by default	Toggle whether the bridge starts automatically with the CLI

Browser Tools

When connected, the agent gains access to these browser tools:

Navigation & Interaction

Tool	Description
browser_navigate	Navigate to a URL
browser_click	Click an element by CSS selector
browser_type	Type text into an input element
browser_press_key	Send a keyboard event (Enter, Escape, etc.)
browser_scroll	Scroll the page or to a specific element

Reading & Inspection

Tool	Description
browser_get_page_context	Get page title, URL, headings, metadata, and body text
browser_get_element	Get computed styles, rect, and attributes of an element
browser_find_element	Find elements by selector, text content, or ARIA role
browser_wait_for_element	Wait for an element to appear (5s timeout)
browser_screenshot	Capture the current page; use save: true to download a PNG
browser_take_full_page_screenshot	Capture the full page; use save: true to download a PNG

Debugging

Tool	Description
browser_read_console	Read captured console messages (errors, warnings, info)
browser_read_network	Read captured network requests with filtering by URL, method, status
browser_get_tabs	List all open browser tabs
browser_get_tab_groups	List tab groups with their member tabs

Tool Examples

> Read the console errors on this page
  → agent calls browser_read_console with level: "error"

> What network requests are failing?
  → agent calls browser_read_network with status: "4"

> Fill in the login form with test@example.com
  → agent calls browser_type with selector: "#email", text: "test@example.com"

> Take a screenshot of the current page
  → agent calls browser_screenshot

Experimental reliable browser tools V2

Browser tools V2 are opt-in and disabled by default. Enable the experiment, then restart the CLI:

autohand experiments enable experimental_browser_tools_v2

The equivalent config is:

{
  "features": {
    "experimentalBrowserToolsV2": true
  }
}

V2 is exposed only after the restarted CLI and extension negotiate protocol version 2 through autohand.browserCapabilities.set. A new CLI paired with an older extension, or an older CLI paired with a new extension, continues to use the legacy browser tools.

Use this flow for reliable targeting:

1. Call browser_snapshot and select the opaque ref for the intended element.
2. Act with that ref, or use a CSS selector or role/name locator only as a compatibility fallback.
3. Use browser_wait_for with an element, text, value, URL, load, or network-idle condition. Waits default to 10 seconds and are capped at 25 seconds.
4. If a ref is reported as stale or ambiguous, take a new snapshot. Refs are scoped to one tab, frame, document, and browser session; never retry an old ref after navigation.

For forms, use browser_inspect_form → browser_fill_form → browser_validate_form → browser_submit_form. Filling is sequential and never submits. Submission validates first, calls requestSubmit once, and can accept a typed post-submit wait. Do not retry a submission whose outcome is ambiguous. browser_reset_form, file upload, form submission, and dialog handling retain the normal interactive approval flow; YOLO and Automode keep their existing approval behavior.

Passwords, one-time codes, payment fields, API keys, and secret-like values are redacted from browser results and tool events. Upload paths are resolved locally by the CLI and results expose basenames only. V2 does not add credential storage, payment autofill, CAPTCHA bypass, arbitrary JavaScript execution, arbitrary sleeps, blind retries, or unrestricted browser URL fetching to the default Chrome policy.

Configuration

Add to ~/.autohand/config.json:

{
  "chrome": {
    "extensionId": "your-extension-id",
    "enabledByDefault": false,
    "browser": "auto",
    "userDataDir": "/path/to/chrome/user-data",
    "profileDirectory": "Default"
  }
}

Key	Type	Default	Description
extensionId	string	—	Chrome extension ID for direct handoff
enabledByDefault	boolean	false	Auto-start browser bridge with CLI
browser	string	"auto"	Preferred browser: auto, chrome, chromium, brave, edge
userDataDir	string	—	Browser user data directory
profileDirectory	string	—	Profile directory name (e.g., "Default")
installUrl	string	—	Fallback URL when extension ID is not set

Connection Lifecycle

Connecting

1. User runs /browser → selects Open in Chrome
2. CLI creates a handoff token in ~/.autohand/chrome/handoffs/
3. Chrome opens, extension attaches to the session via the token
4. Native messaging bridge forwards JSON-RPC between CLI and extension

Disconnecting

The connection can be closed from either side:

From CLI:

/browser disconnect

From extension: Click the disconnect button in the side panel. Closing the panel only detaches the UI; the background bridge keeps an in-flight browser tool connected until it finishes or the CLI/browser session ends.

Reconnecting

If the connection drops (CLI crash, browser restart, etc.):

1. The extension shows a Connection lost banner with the reason
2. Auto-reconnect attempts with exponential backoff (1s, 2s, 4s... up to 15s, max 10 attempts)
3. Manual retry via the banner's retry button or the header reconnect icon
4. Re-focusing the side panel also triggers a reconnect attempt

From CLI: Run /browser again and select Open in Chrome to create a new handoff.

Heartbeat

The extension sends a health check every 30 seconds. If the CLI doesn't respond within 10 seconds, the connection is marked as lost and auto-reconnect begins.

Architecture

Chrome Extension                          CLI Process
┌──────────────┐                    ┌──────────────────┐
│  Side Panel  │◄──── Chrome ─────►│  Native Host      │
│  (UI + RPC)  │      Native       │  (host.js)        │
│              │      Messaging    │       │            │
│  Content     │      (4-byte LE   │       ▼            │
│  Script      │       + JSON)     │  autohand          │
│  (DOM tools) │                   │  --mode rpc        │
└──────────────┘                    │  (JSON-RPC 2.0)   │
                                    └──────────────────┘

• Side Panel: Main UI, sends prompts and receives responses
• Content Script: Runs on every page, executes DOM tools (click, type, find, etc.)
• Background Worker: Routes messages, handles context menus and shortcuts
• Native Host: Node.js bridge that translates Chrome native messaging to stdio
• CLI RPC Mode: The agent running in JSON-RPC server mode

Permissions

Browser tool permissions follow the CLI's permission mode:

Mode	Behavior
Interactive	Agent asks before each browser action
Full-auto	Agent acts without asking
Restricted	Agent denies dangerous operations

Site-level permissions are inherited from the Chrome extension's host permissions.

Troubleshooting

"Not installed" status

The native messaging host is not installed. Run /browser and select Reconnect extension, or:

autohand --browser

"Disconnected" status

The CLI is running but no active handoff exists. Run /browser → Open in Chrome to create one.

Extension can't find the CLI

Make sure autohand is in your PATH, or set cliPath in the extension settings to the full path of the binary.

Port conflicts

The OAuth callback server uses port 1455. If another process is using it:

lsof -i :1455
kill <PID>