README.md

Mote implements the ReAct (Reason + Act) paradigm, ensuring the agent doesn't blindly click around, but follows a structured loop:

Perception → Reasoning → Action

┌─────────────────────────────────────────────────────────────┐
│                           Mote                              │
│               (Playwright-based Orchestrator)               │
│       Entry Point · Lifecycle · State Management            │
└───────────────┬───────────────────────┬─────────────────────┘
                │                       ▲
                │  Observe              │
                ▼                       │
┌───────────────────────────────────┐   │ ┌─────────────────────────────────────┐
│           EYE LAYER               │   │ │      HUMAN-IN-THE-LOOP              │
│    (Perception & Cleansing)       │   │ │ (Safety Guard & Final Decision)     │
├───────────────────────────────────┤   │ ├─────────────────────────────────────┤
│ • Clean HTML Content Extraction   │   │ │ • Monitor & Observe Agent View      │
│ • Action Element Map              │   │ │ • Intervene & Correct Actions       │
│ • Markdown Conversion             │   │ │ • Join Decision-Making at Any Stage │
└──────────────────┬────────────────┘   │ └─────────────────────────────────────┘
                 ▲ │                    │
 Refinement Loop │ │ Perception Payload │
  (as needed)    │ ▼                    │
┌───────────────────────────────────┐   │
│           BRAIN LAYER             │   │
│       (Planning & Reasoning)      │   │
├───────────────────────────────────┤   │
│ • Intent Understanding            │   │
│ • Action Planning (Steps)         │   │
│ • Tool / Skill Selection          │   │
│ • Outcome Evaluation              │   │
│ • Self-Correction & Re-planning   │   │
└───────────────┬───────────────────┘   │
             ▲  │                       │
State Update │  │ Propose Plan          │
(Memory Feed)│  │                       │
             │  ▼                       │
┌───────────────────────────────────┐   │
│           HAND LAYER              │   │
│        (Action Execution)         │   │
├───────────────────────────────────┤   │
│ • Execute via Built-in Skills     │   │
│ • Real-time Status Reporting      │   │
│ • Step-by-Step Execution          │   │
│ • Continuous State Update         │   │
└───────────────┬───────────────────┘   │
                │                       │
                │  Execute / Save       │
                │                       │
                ▼                       │
┌───────────────────────────────────┐   │
│     Execution & Evidence Store    │   │
│   (Data Persistence & Security)   │   │
├───────────────────────────────────┤   │
│ • System Logs & Session Traces    │   │
│ • Visual Evidence (Screenshots)   │   │
│ • Execution Outputs (PDF/JSON)    │   │
│ • Memory (Execution Context)      │   │
└───────────────────────────────────┘   │
                │                       │
                │                       │
                └────────── Loop ───────┘
               （Next-session optimization）

Core Logic

ReAct Loop

Mote observes the page through the Analyzer, reasons via the Brain, and executes actions using Skills. This loop continues until the goal is achieved.

Tree of Thoughts (ToT) Influence

The Human-in-the-Loop acts as a manual path evaluator. When the agent reaches a complex decision branch, users can prune incorrect paths, guiding the agent toward the most efficient reasoning tree.

mote/
├── src/
│   ├── mote.ts                  # Entry point: orchestrates the state machine
│   ├── browser.ts               # Playwright browser context management
│   ├── observe.ts               # Perception: page sweep, element extraction, markdown generation
│   ├── reason.ts                # Reasoning: LLM-based action planning
│   ├── prompt.ts                # System/user prompt templates for LLM
│   ├── interaction.ts           # Human-in-the-loop intervention system
│   │
│   ├── act/                     # Action execution layer
│   │   ├── act.ts               # executeAction: click, type, scroll, navigate, etc.
│   │   ├── helpers.ts           # humanMouseMove, humanType, humanDelay
│   │   └── element-resolution.ts # verifyOrResolve, fuzzyMatchElement
│   │
│   ├── handlers/                # State machine handlers
│   │   ├── setup.ts             # SETUP: initialization and setup steps
│   │   ├── cycle.ts             # CYCLE_START / CYCLE_END: cycle lifecycle
│   │   ├── observe.ts           # OBSERVE: page perception
│   │   ├── reason.ts            # REASON: blueprint execution and LLM reasoning
│   │   ├── act.ts               # ACT: action execution and pointer advancement
│   │   └── wrapup.ts            # WRAPUP: session finalization
│   │
│   ├── bootstrap/               # Session bootstrapping
│   │   ├── bootstrap.ts         # Initialize session from preset or goal
│   │   └── plan-preview.ts      # Preview session plan before execution
│   │
│   ├── config/                  # Configuration
│   │   ├── defaults.ts          # Default settings
│   │   └── resolver.ts          # Config resolution and merging
│   │
│   ├── recorder/                # Workflow recorder
│   │   ├── runtime.ts           # Recorder runtime loop
│   │   ├── page-observer.ts     # DOM observation and element tracking
│   │   ├── interceptor.ts       # Network/action interception
│   │   ├── llm-generator.ts     # Generate session plans from recordings
│   │   └── handlers/            # Recording state handlers
│   │
│   ├── runtime/                 # Agent runtime
│   │   ├── runtime.ts           # Runtime state and context management
│   │   └── result.ts            # Session result formatting
│   │
│   ├── types/                   # TypeScript type definitions
│   │   ├── actions.ts           # Action, StepPlan, SessionPlan types
│   │   ├── config.ts            # Configuration types
│   │   ├── context.ts           # AgentContext, RuntimeState
│   │   ├── page.ts              # PageState, Element types
│   │   ├── session.ts           # Session tracker, cycle management
│   │   ├── state-machine.ts     # State machine phase definitions
│   │   ├── services.ts          # Service interfaces
│   │   └── results.ts           # Result types
│   │
│   └── utils/                   # Utilities
│       ├── checkpoint.ts        # Session checkpoint save/restore
│       ├── debug.ts             # Debug logging helpers
│       ├── handler-helpers.ts   # Shared handler utilities
│       ├── preset.ts            # Preset loading, prompt rendering
│       └── verification.ts      # Script/LLM-based verification
│
├── presets/                     # Automation presets
│   ├── _template/               # Template for creating new presets
│   └── percipio-quiz-helper/    # Example: auto-complete quizzes
│       ├── preset.json          # Preset metadata and settings
│       ├── session-plan.json    # Step-by-step execution plan
│       └── prompts/             # Step-specific LLM prompts
│
├── tests/                       # Test suite (vitest)
├── data/                        # Session logs and traces
├── docs/                        # Documentation
├── package.json
├── tsconfig.json
├── vitest.config.ts
└── README.md

Presets are reusable automation packages that bundle a task definition, execution blueprint, and configuration overrides into a single folder. They let you capture a workflow once and replay it reliably.

Preset folder structure

Each preset lives under presets/{name}/:

presets/percipio-quiz-helper/
├── preset.json              # Task definition and settings
├── session-plan.json        # Step-by-step execution blueprint
├── system-prompt.md         # Custom LLM system prompt (optional)
└── prompts/                 # Step-specific LLM prompts (optional)
    ├── step-1-choose-quiz.md
    └── step-3-answer-question.md

Folders starting with _ or . (like _template/) are ignored when listing presets.

preset.json

Defines what the preset does and how the agent should behave:

{
  "name": "percipio-quiz-helper",
  "description": "Complete all quizzes until every test shows 'Completed' status",
  "goal": {
    "name": "Complete all quizzes",
    "description": "Navigate through each quiz and answer all questions",
    "context": {}
  },
  "sessionPlanRef": "./session-plan.json",
  "engagementMode": "autonomous",
  "maxSteps": -1
}

A preset can override any configurable setting — browser options (headless, stealth, slowMo), timeouts, LLM provider/model, token limits, engagement mode, and step limits. These override defaults and environment variables but can still be overridden programmatically at runtime.

Config priority: CLI/programmatic overrides > preset values > environment variables > defaults.

Session plan (the execution blueprint)

The session plan is the core of a preset. It defines a structured execution flow:

setup → [cycleStart → cycle → cycleEnd] × N → wrapup → verification

• Setup steps run once at the start (e.g., navigate to a page, log in)
• Cycle steps repeat N times (or indefinitely with numberOfCycles: -1)
• Wrapup steps run once at the end

Each step can be:

Semantic selectors like {desc:"the Submit button"} tell the LLM to find the right element by description rather than a fixed CSS selector, making presets resilient to UI changes.

Loops allow repeating a group of steps within a cycle — either a fixed number of iterations or until a JavaScript condition evaluates to true.

Verification scripts are JavaScript functions that run after phases (setup, cycle, wrapup) to confirm success. On failure, the agent can retry with LLM assistance, continue, or abort.

Wait-for-ready hooks handle async content (SPAs, lazy-loaded pages) by running a JavaScript promise that resolves when the page is ready before the step executes.

Step prompts

Step prompts are Markdown files that give the LLM domain-specific guidance for individual steps. They support template variables:

• {{instruction}} — the step's instruction field
• {{context.KEY}} — values from goal.context
• {{goal}} — the full goal object

This lets you inject runtime context (e.g., credentials, search terms) into step-level reasoning without hardcoding them.

Engagement modes

Presets can set the level of human oversight:

Creating a new preset

1. Copy presets/_template/ to presets/your-preset-name/
2. Edit preset.json with your goal and settings
3. Define your session plan in session-plan.json
4. Add step prompts under prompts/ as needed
5. Run with npm start and select your preset

Alternatively, use the recorder to capture a workflow interactively — it generates a session plan from your actions automatically.