AXorcist 🧙‍♂️ - The power of Swift compels your UI to obey!

Swift wrapper for macOS Accessibility—chainable, fuzzy-matched queries
that read, click, and inspect any UI. The dark arts meet modern Swift!

---

Platform target: macOS 14.0 and later. AXorcist sits on top of the Accessibility APIs that only ship on macOS, so CI and releases intentionally stay macOS-only.

AXorcist harnesses the supernatural powers of macOS Accessibility APIs to give you mystical control over any application's interface. Whether you're automating workflows, testing applications, or building assistive technologies, AXorcist provides the incantations you need to make UI elements bend to your will.

Overview

AXorcist enables developers to create sophisticated automation tools, testing frameworks, and accessibility utilities by providing:

• Type-safe API: Compile-time safety for accessibility attributes and operations
• Modern Swift Patterns: Async/await, structured concurrency, and error handling
• Comprehensive Command System: Query, action, observation, and batch operations
• Element Management: Efficient UI element discovery and interaction
• Permission Handling: Streamlined accessibility permission workflows

---

The examples below cover the Swift API and CLI. In Xcode, use Product → Build Documentation to browse the DocC reference.

Core Classes Reference

AXorcist (Main Class)

The central orchestrator for all accessibility operations.

@MainActor
public class AXorcist {
    static let shared = AXorcist()
    public func runCommand(_ commandEnvelope: AXCommandEnvelope) -> AXResponse
}

Key Features:

• Singleton pattern for consistent state management
• Command-based architecture for all operations
• MainActor isolation for thread safety
• Comprehensive error handling

Usage Example:

import AXorcist

let axorcist = AXorcist.shared
let query = QueryCommand(
    appIdentifier: "Safari",
    locator: Locator(criteria: [Criterion(attribute: "AXRole", value: "AXButton")]))
let command = AXCommandEnvelope(
    commandID: "find-button",
    command: .query(query)
)
let response = axorcist.runCommand(command)

Element

Swift wrapper around AXUIElement providing modern API patterns.

public struct Element: Equatable, Hashable {
    public let underlyingElement: AXUIElement
    public var attributes: [String: AttributeValue]?
    public var prefetchedChildren: [Element]?
    public var actions: [String]?
}

Key Features:

• Type-safe property access with computed properties
• Automatic value conversion between CF and Swift types
• Hierarchy navigation with caching support
• Action execution with error handling
• Batch attribute fetching for performance

Common Operations: getElementAttributes(...) applies valueFormatOption to parent, child and focused-element descriptions and to native values rendered as text.

// Create element wrapper
let element = Element(axUIElement)

// Access properties safely
let title = element.title()
let role = element.role()
let isEnabled = element.isEnabled()

// Perform a native action
try element.performAction(.press)

// Set the native value attribute
try element.setValue("Hello World")

// Navigate hierarchy
let children = element.children()
let parent = element.parent()

JSON encoding preserves numeric attribute values as numbers, including 0 and 1; Boolean attributes remain true or false. AnyCodable decodes unsigned integers above Int.max as UInt64 to retain their exact value. The value-formatting helpers display geometry and text ranges using their native types.

Integer text in geometry/range parsing accepts the full signed range, including Int.min. Overflow and malformed signs fail parsing without advancing the numeric cursor.

Floating-point scanning converts complete scientific-notation tokens, retaining representable subnormal values and signed zero. Native attribute setters preserve numeric NSNumber values, including 0 and 1, as CFNumber values; Boolean inputs remain CFBoolean values.

AXPermissionHelpers

Modern async/await API for accessibility permissions.

public struct AXPermissionHelpers {
    static func hasAccessibilityPermissions() -> Bool
    static func requestPermissions() async -> Bool
    static func permissionChanges(interval: TimeInterval = 1.0) -> AsyncStream<Bool>
    static func isSandboxed() -> Bool
}

Key Features:

• Async/await permission handling
• Real-time permission monitoring with AsyncStream
• Sandbox detection for permission strategy
• Non-blocking permission requests

Usage Patterns:

// Check current status
let hasPermissions = AXPermissionHelpers.hasAccessibilityPermissions()

// Request permissions asynchronously
let granted = await AXPermissionHelpers.requestPermissions()

// Monitor permission changes
for await hasPermissions in AXPermissionHelpers.permissionChanges() {
    if hasPermissions {
        print("Permissions granted!")
        // Enable accessibility features
    } else {
        print("Permissions revoked!")
        // Disable accessibility features
    }
}

Table of Contents

• Features
• Installation
• Quick Start
• Element Search and Matching
• Available Commands
• Actions
• Notifications and Observing
• Command-Line Usage
• Advanced Examples
• Architecture
• Troubleshooting

Features

• 🔍 Powerful Search: Find UI elements using multiple criteria with flexible matching
• 🎯 Precise Navigation: Navigate UI hierarchies with path-based locators
• 🎬 Actions: Perform clicks, set values, and trigger UI interactions
• 👁️ Observation: Monitor UI changes in real-time with notifications
• 🚀 Batch Operations: Execute multiple commands efficiently
• 📊 Rich Attributes: Access all accessibility attributes and computed properties
• 🔧 CLI Tool: Full command-line interface for scripting and automation
• 📝 Comprehensive Logging: Debug support with detailed operation logs

Installation

Swift Package Manager

Add to your Package.swift:

dependencies: [
    .package(url: "https://github.com/openclaw/AXorcist.git", from: "0.1.11")
]

Command Line Tool

Install the signed, notarized universal CLI with Homebrew:

brew install openclaw/tap/axorc

Or build and install it from source:

swift build -c release --product axorc
install -m 755 .build/release/axorc /usr/local/bin/axorc

Run axorc permissions after installation. macOS will need Accessibility permission for inspection and automation.

Maintainers: see docs/releasing.md for the artifact and tap workflow.

Local Development

AXorcist always declares the remote Commander dependency at exactly 0.2.4, regardless of checkout location or sibling folders. To work on a sibling Commander checkout, explicitly override it from your root workspace (the AXorcist checkout, or the package consuming AXorcist):

swift package resolve
swift package edit Commander --path ../Commander
# Develop against the local checkout, then restore the released dependency:
swift package unedit Commander

The edit belongs to that workspace and leaves AXorcist's manifest unchanged. A consuming package can also explicitly add .package(path: "../Commander") to its root manifest's dependencies; remove that entry to restore versioned resolution. Keep these local overrides out of published manifests.

Run make test-commander-dependency for offline manifest and dependency-graph regression checks. They use disposable Git fixtures and isolated SwiftPM configuration and caches, including default and custom scratch paths, without building or running the app.

Quick Start

Swift API

import AXorcist

// Initialize AXorcist
let axorcist = AXorcist()

// Create a query command
let query = QueryCommand(
    appIdentifier: "com.apple.TextEdit",
    locator: Locator(criteria: [
        Criterion(attribute: "AXRole", value: "AXTextArea")
    ]),
    attributesToReturn: ["AXValue", "AXRole"]
)

// Execute the command
let response = axorcist.runCommand(AXCommandEnvelope(
    commandID: "query-1",
    command: .query(query)
))

Command Line

# Print a shallow accessibility tree
axorc tree --app Safari --depth 3

# Find the Back button
axorc find --app Safari --role AXButton --title Back

# Use the full JSON protocol for actions and advanced queries
echo '{"command_id":"back","command":"performAction","application":"Safari","locator":{"criteria":[{"attribute":"AXTitle","value":"Back"}]},"action_name":"AXPress"}' | axorc raw --stdin

Element Search and Matching

Matching Types

AXorcist supports multiple matching strategies:

• exact - Exact string match (default)
• contains - Case-insensitive substring match
• regex - Regular expression match
• containsAny - Matches if any comma-separated value is contained
• prefix - String starts with the expected value
• suffix - String ends with the expected value

Searchable Attributes

Core Attributes

• role / AXRole - Element's role (e.g., "AXButton", "AXWindow")
• subrole / AXSubrole - Additional role information
• identifier / id / AXIdentifier - Developer-assigned unique ID
• title / AXTitle - Element's title
• value / AXValue - Element's value
• description / AXDescription - Detailed description
• help / AXHelp - Tooltip/help text
• placeholder / AXPlaceholderValue - Placeholder text

State Attributes

• enabled / AXEnabled - Is element enabled?
• focused / AXFocused - Is element focused?
• hidden / AXHidden - Is element hidden?
• busy / AXElementBusy - Is element busy?

Special Attributes

• pid - Process ID (exact match only)
• domclasslist / AXDOMClassList - Web element classes
• domid / AXDOMIdentifier - DOM element ID
• computedname / name - Computed accessible name

Search Examples

Find button by exact title

{
  "criteria": [
    {"attribute": "role", "value": "AXButton"},
    {"attribute": "title", "value": "Submit"}
  ]
}

Find text field containing "email"

{
  "criteria": [
    {"attribute": "role", "value": "AXTextField"},
    {"attribute": "title", "value": "email", "match_type": "contains"}
  ]
}

Find element by multiple classes (web content)

{
  "criteria": [
    {"attribute": "domclasslist", "value": "btn-primary", "match_type": "contains"}
  ]
}

Using OR logic

{
  "criteria": [
    {"attribute": "title", "value": "Save"},
    {"attribute": "title", "value": "Submit"},
    {"attribute": "title", "value": "OK"}
  ],
  "matchAll": false
}

Path Navigation

Navigate through UI hierarchies with path hints:

{
  "path_from_root": [
    {"attribute": "role", "value": "AXWindow", "depth": 1},
    {"attribute": "identifier", "value": "main-content", "depth": 3},
    {"attribute": "role", "value": "AXButton"}
  ]
}

Each path component supports:

• attribute - What to match
• value - Expected value
• depth - Max search depth for this step (default: 3)
• match_type - How to match (default: exact)

Available Commands

1. Query

Find elements and retrieve their attributes.

{
  "command_id": "find-text-area",
  "command": "query",
  "application": "com.apple.TextEdit",
  "locator": {
    "criteria": [{"attribute": "AXRole", "value": "AXTextArea"}]
  },
  "attributes": ["AXValue", "AXRole", "AXTitle"],
  "max_depth": 10
}

2. Perform Action

Execute actions on elements.

{
  "command_id": "press-back",
  "command": "performAction",
  "application": "Safari",
  "locator": {
    "criteria": [{"attribute": "AXTitle", "value": "Back"}]
  },
  "action_name": "AXPress"
}

3. Get Focused Element

Retrieve the currently focused element.

{
  "command_id": "focused-element",
  "command": "getFocusedElement",
  "attributes": ["AXRole", "AXTitle", "AXValue"]
}

4. Get Element at Point

Find element at specific screen coordinates.

{
  "command_id": "element-at-point",
  "command": "getElementAtPoint",
  "point": [500, 300],
  "attributes": ["AXRole", "AXTitle"]
}

5. Batch Commands

Execute multiple commands in sequence.

Every child must convert to a supported library command before execution starts, including children of nested batches. Invalid children reject the entire batch and no children execute; callers that previously relied on invalid children being skipped must correct those requests. After validation, runtime failures retain the existing behavior: remaining children still execute, and the response reports an aggregate error without per-command data. Encoding failures return escaped JSON even when command IDs contain quotes or newlines.

{
  "command_id": "inspect-and-fill",
  "command": "batch",
  "sub_commands": [
    {
      "command_id": "find-text-area",
      "command": "query",
      "application": "TextEdit",
      "locator": {"criteria": [{"attribute": "AXRole", "value": "AXTextArea"}]}
    },
    {
      "command_id": "fill-text-area",
      "command": "setFocusedValue",
      "application": "TextEdit",
      "locator": {"criteria": [{"attribute": "AXRole", "value": "AXTextArea"}]},
      "action_value": "Hello, World!"
    }
  ]
}

6. Observe Notifications

Monitor UI changes in real-time.

{
  "command_id": "watch-text-edit",
  "command": "observe",
  "application": "com.apple.TextEdit",
  "notifications": ["AXValueChanged", "AXFocusedUIElementChanged"],
  "include_element_details": ["AXRole", "AXTitle", "AXValue"],
  "watch_children": false
}

7. Collect All

Recursively collect all elements.

{
  "command_id": "collect-buttons",
  "command": "collectAll",
  "application": "Safari",
  "attributes": ["AXRole", "AXTitle"],
  "max_depth": 5,
  "filter_criteria": {"AXRole": "AXButton"}
}

Actions

Swift scrolling helpers throw UIAutomationError.invalidScrollAmount before posting an event for invalid inputs. InputDriver.scroll requires finite deltas whose line counts (division by 10, truncated toward zero) fit the native event fields. Element.scrollAt requires a representable signed amount for nonsmooth scrolling and a nonnegative step count for smooth scrolling; a zero smooth count is a no-op. Both helpers verify the native event retains the requested delta, rejecting values that macOS would silently wrap. Representable signed nonsmooth amounts retain their existing direction behavior.

Available actions to perform on elements:

• AXPress - Click/activate an element
• AXIncrement - Increment value (sliders, steppers)
• AXDecrement - Decrement value
• AXConfirm - Confirm action
• AXCancel - Cancel action
• AXShowMenu - Show context menu
• AXPick - Pick/select element
• AXRaise - Bring element to front

Setting Text Values

Setting AXValue is an attribute mutation, not a native accessibility action. Use setFocusedValue when the target may need focus:

{
  "command_id": "replace-text",
  "command": "setFocusedValue",
  "application": "TextEdit",
  "locator": {"criteria": [{"attribute": "AXRole", "value": "AXTextArea"}]},
  "action_value": "New text content"
}

The published performAction spelling with "action_name": "AXSetValue" remains a compatibility alias. It requires a string action_value and writes AXValue directly without invoking an accessibility action or changing focus.

The instance methods element.typeText(...) and element.clearField() establish native focus before posting keyboard input and throw if focus cannot be established. Clearing as part of typeText(..., clearFirst: true) uses the same preflight.

Notifications and Observing

Monitor UI changes with these notifications:

• AXFocusedUIElementChanged - Focus changes
• AXValueChanged - Value changes
• AXUIElementDestroyed - Element destruction
• AXWindowCreated - Window creation
• AXWindowResized - Window resizing
• AXTitleChanged - Title changes
• AXSelectedTextChanged - Text selection changes
• AXLayoutChanged - Layout updates

Observe and stopObservation commands executed by the same AXorcist instance share one subscription registry, so a successful stop clears the observations that instance started.

Accessibility observers are application-scoped on macOS; PID 0 and the system-wide AX element cannot receive notifications. NotificationWatcher(globalNotification:) implements global watching by registering one observer for each running user application and observing native KVO changes to NSWorkspace.runningApplications to keep that set current, including menu-bar agents and background applications:

let watcher = NotificationWatcher(globalNotification: .focusedUIElementChanged) {
    pid, notification, element, userInfo in
    print("\(pid): \(notification.rawValue)")
}
try watcher.start()

Complete workspace snapshots are reconciled in order after KVO delivery. PID and launch-readiness reads, readiness subscription, and its cleanup run on one serial background queue; a blocked metadata read leaves the main actor and stop responsive. Membership supplies termination state without querying isTerminated. Session and membership generations discard late results after stop, restart, or replacement, including removal and re-addition of the same application. A blocked native read can delay subsequent metadata work until it returns; it does not spawn extra workers. Each readiness observation retains its exact application wrapper until invalidation completes, including queued cleanup after stop or wrapper replacement.

Applications that do not support the requested notification are skipped. Starting installs lifecycle tracking and returns without waiting for per-application Accessibility endpoints; observer creation, registration, and cleanup run off the main actor with bounded native messaging timeouts, so one wedged app cannot block startup or teardown. The source-compatible nil-PID AXObserverCenter.subscribe entry point returns an explicit setup failure instead of attempting to construct an invalid PID-zero observer. A transient registration failure after an application lifecycle event receives three bounded retries over 10.5 seconds, and an isFinishedLaunching readiness transition triggers an immediate fresh attempt. Termination cancels pending registration and retry work, so this recovery never becomes a polling loop.

Observer Example

{
  "command_id": "watch-text",
  "command": "observe",
  "application": "TextEdit",
  "notifications": ["AXValueChanged", "AXFocusedUIElementChanged"],
  "locator": {"criteria": [{"attribute": "AXRole", "value": "AXTextArea"}]},
  "include_element_details": ["AXRole", "AXTitle", "AXValue"]
}

Command-Line Usage

axorc has human-readable inspection commands and a stable JSON mode for scripts and advanced automation.

Inspect Applications

# Check permission and recovery instructions
axorc permissions

# Print a hierarchy; use a bundle identifier when names are ambiguous
axorc tree --app com.apple.dock --depth 3

# Limit a tree to one role and emit JSON for scripts
axorc tree --app com.apple.dock --role AXDockItem --json

# Find one element with exact matching
axorc find --app Safari --role AXButton --title Back

# Use case-insensitive substring matching
axorc find --app Safari --title address --contains

Run axorc --help or axorc help find for the complete terminal reference. Human-readable output goes to stdout, diagnostics go to stderr, and failures return nonzero exit codes.

JSON Protocol

Every JSON command requires command_id and command. JSON command names and fields differ from the human-readable CLI:

CLI	JSON protocol
tree --app <app> --depth 3	"command":"collectAll", "application":"<app>", "max_depth":3
find --app <app> --role AXButton	"command":"query", "application":"<app>", "locator":{"criteria":[{"attribute":"AXRole","value":"AXButton"}]}

tree and find are not JSON command names. Use application and max_depth, not app and depth. For example, the raw equivalent of axorc tree --app com.apple.mail --depth 3 --json is:

axorc raw --json '{"command_id":"mail-tree","command":"collectAll","application":"com.apple.mail","max_depth":3,"attributes":["AXRole","AXTitle","AXDescription","AXIdentifier","AXValue"]}'

Protocol commands are ping, query, getAttributes, describeElement, getElementAtPoint, getFocusedElement, performAction, batch, observe, collectAll, stopObservation, isProcessTrusted, isAXFeatureEnabled, setFocusedValue, and extractText. The reserved names setNotificationHandler, removeNotificationHandler, and getElementDescription decode but return a not-implemented error.

Locators may contain criteria, path_from_root, or both; omitted criteria defaults to an empty list. Invalid payloads return a nonzero exit code and a JSON error with the failing field path. An application-not-found or Accessibility error means decoding succeeded and the command reached execution.

Input can come from standard input, a file, an argument, or the legacy root-level syntax:

# Standard input
echo '{
  "command_id": "enabled-button",
  "command": "query",
  "application": "Safari",
  "locator": {
    "criteria": [
      {"attribute": "AXRole", "value": "AXButton"},
      {"attribute": "AXEnabled", "value": "true"}
    ]
  }
}' | axorc raw --stdin

# File
axorc raw --file command.json

# Argument
axorc raw --json '{"command_id":"health","command":"ping"}'

# Action using path navigation
echo '{
  "command_id": "press-back",
  "command": "performAction",
  "application": "com.apple.Safari",
  "locator": {
    "path_from_root": [
      {"attribute": "AXRole", "value": "AXWindow"},
      {"attribute": "AXIdentifier", "value": "toolbar"}
    ],
    "criteria": [{"attribute": "AXTitle", "value": "Back"}]
  },
  "action_name": "AXPress"
}' | axorc raw --stdin

Existing invocations such as axorc --stdin and axorc '{...}' remain supported. Prefer the explicit raw subcommand in new scripts.

Advanced Examples

Complex Search with Path Navigation

{
  "command_id": "find-submit",
  "command": "query",
  "application": "com.apple.Safari",
  "locator": {
    "path_from_root": [
      {"attribute": "AXRole", "value": "AXWindow", "depth": 1},
      {"attribute": "AXRole", "value": "AXWebArea", "depth": 5}
    ],
    "criteria": [
      {"attribute": "AXRole", "value": "AXButton"},
      {"attribute": "AXDOMClassList", "value": "submit-button primary", "match_type": "contains"}
    ]
  },
  "attributes": ["AXTitle", "AXValue", "AXEnabled", "AXPosition", "AXSize"]
}

Automated Form Filling

{
  "command_id": "fill-form",
  "command": "batch",
  "sub_commands": [
    {
      "command_id": "fill-email",
      "command": "setFocusedValue",
      "application": "Safari",
      "locator": {
        "criteria": [
          {"attribute": "AXRole", "value": "AXTextField"},
          {"attribute": "AXPlaceholderValue", "value": "Email", "match_type": "contains"}
        ]
      },
      "action_value": "user@example.com"
    },
    {
      "command_id": "fill-password",
      "command": "setFocusedValue",
      "application": "Safari",
      "locator": {
        "criteria": [
          {"attribute": "AXRole", "value": "AXTextField"},
          {"attribute": "AXPlaceholderValue", "value": "Password", "match_type": "contains"}
        ]
      },
      "action_value": "example-value"
    },
    {
      "command_id": "submit-form",
      "command": "performAction",
      "application": "Safari",
      "locator": {
        "criteria": [
          {"attribute": "AXRole", "value": "AXButton"},
          {"attribute": "AXTitle", "value": "Sign In", "match_type": "contains"}
        ]
      },
      "action_name": "AXPress"
    }
  ]
}

Monitoring Text Changes

{
  "command_id": "watch-text",
  "command": "observe",
  "application": "com.apple.TextEdit",
  "notifications": ["AXValueChanged", "AXSelectedTextChanged"],
  "locator": {
    "criteria": [{"attribute": "AXRole", "value": "AXTextArea"}]
  },
  "include_element_details": ["AXRole", "AXTitle", "AXValue"],
  "watch_children": true
}

Architecture

Core Components

• AXorcist - Main orchestrator class
• Element - Wrapper around AXUIElement with convenience methods
• ElementSearch - Tree traversal and matching engine
• Criterion matching functions - Attribute and string comparison
• Path navigation functions - Hierarchical and JSON path resolution
• AXObserverCenter - Notification management

Thread Safety

Accessibility element operations are MainActor-isolated. Native observer work runs on bounded background workers.

Global logging and clearing helpers enqueue requests on the main actor. axGetLogEntries and axGetLogsAsStrings return immutable snapshots of processed history. Main-actor reads through GlobalAXLogger.shared drain pending requests first; the CLI uses this path before encoding debug responses. Configuration and payload formatting remain on the main actor. The minimal detail level keeps errors and critical messages, and clearing history resets duplicate suppression.

Global entry snapshots copy details through JSON so they never retain caller-owned reference objects; unencodable details are omitted from those entries. Raw payloads remain available through main-actor GlobalAXLogger.shared.getEntries(), and JSON formatting reports encoding failures explicitly.

Queued metadata retains AnyCodable's existing contract: callers must keep referenced values immutable or synchronized while they are in flight. Formatting reads those values on the main actor when the request is processed.

AXTimeoutHelper.withTimeout runs its async operation concurrently and returns the first result, timeout, or caller cancellation without waiting for uncooperative work to finish. Cancellation received before the call starts is preserved as CancellationError. Timed-out or cancelled work may continue in the background; the helper does not undo its effects. Use Element.withMessagingTimeout to bound synchronous native Accessibility messages.

Performance Optimizations

• Early termination on first match
• Depth-limited searches
• Efficient tree traversal with visitor pattern
• Caching of frequently accessed attributes

Troubleshooting

Permission Issues

Check Accessibility permission and print recovery instructions:

axorc permissions

Finding Elements

Use the debug flag to see detailed search logs:

axorc raw --file command.json --debug

Common Issues

1. Element not found: Try broader criteria or increase search depth
2. Action failed: Ensure element is enabled and supports the action
3. Observer not working: Check notification names and app identifier

Debug Mode

Enable debug logging in commands:

{
  "command_id": "debug-query",
  "command": "query",
  "debug_logging": true,
  ...
}

License

AXorcist is released under the MIT License. See LICENSE for details.

Contributing

Please follow the main Peekaboo contributing guidelines and open pull requests against this repository when proposing AXorcist changes.

Development checks

Run swift test for the safe suites and make check for formatting, linting, native API policy, dependency resolution, and packaging-mode checks. CI uses the tool versions pinned in .github/workflows/ci.yml and scripts/install-validation-tools.sh.

CI tests Swift 6.2.4 with Xcode 16.4 on macOS 15 and Swift 6.3.3 with Xcode 26.6 on macOS 26. Formatting, dependency-fixture and universal-packaging checks run once on the minimum-toolchain job. The repository's CodeQL workflow builds the Swift library and CLI directly with the minimum toolchain and also scans Python and Actions; it replaces GitHub's default setup.

Automation suites are opt-in (RUN_AUTOMATION_TESTS=true swift test) and require Accessibility permission and an interactive desktop. They launch and manipulate TextEdit; use a disposable account or VM. Generate coverage for the selected suites with swift test --enable-code-coverage.