README.md

May 20, 2026 · View on GitHub

Desktop AI Agent Framework · Multi-tentacle Collaboration

🌟 Like an octopus, handle multiple things at once 🌟

🎬 Demo Videos

https://github.com/user-attachments/assets/29a64b38-3f98-4cbc-99d4-662f55cbec74

Text-to-Speech Demo

https://github.com/user-attachments/assets/ef0af274-e988-436f-a7da-a007e1a814ee

WeChat Channel Demo

https://github.com/user-attachments/assets/1de4e3d3-3397-46f8-a6b5-8f9dfef2b580

✨ Core Features

🚀 One-Click Deploy No server, no YAML

⚡ Double-click to install 🐍 Embedded Python env 💾 Portable USB mode 🔒 Data stays local

💰 Cost Transparency Know what you spend

📊 Real-time token counter 📈 Visual cost charts ⚠️ Budget alerts 🔄 Model cost compare

🧩 Markdown Skills Extend without coding

📝 Write SKILL.md 🔗 MCP protocol support 📦 Git install extensions ♻️ Hot-reload enabled

🔄 Visual Workflow Build AI pipelines

🎨 Drag-and-drop editor 🧩 24 node types 📋 Version management 🔍 Run trace & debug

🤖 Visual SubAgent Create AI workers

🎨 GUI agent creator 📁 Isolated workspaces 🎯 Auto task dispatch 🧠 Own config & memory

📚 Knowledge Base Your second brain

📄 Multi-format documents 📝 Markdown notes 🕸️ Knowledge graph 🧠 AI-powered distillation

📡 Multi-Channel Chat everywhere

💬 Desktop / WeChat 🐦 Slack / Discord ✈️ Telegram / DingTalk 📧 Email / Webhook

⏰ Smart Tasks Actually run tasks

▶️ SubAgent execution 📅 Cron/interval/once 💪 Survive restarts 💬 Access context

🗂️ Project Isolation Separate workspaces

⚙️ Per-project config 🔄 Switch instantly 👥 Export for team 💬 Never lose history

🔊 Text-to-Speech Voice your AI

🗣️ Multiple TTS engines 🎵 Natural voice output ⚙️ Customizable settings 📱 Real-time playback

🧠 Observation & Memory Learn from experience

🔍 9 observation types 📝 Auto-extract insights 💾 Promote to memory 👤 User profile tracking

📄 Multi-format Files Read any document

📑 PDF / DOCX / XLSX 📊 PPTX preview 🖼️ Image understanding 🗜️ Context compression

🔄 Visual Workflow

Build complex AI pipelines with a drag-and-drop editor powered by ReactFlow:

Node Types (24 kinds)

Category	Nodes
Flow	Workflow Start, Answer, Workflow End
AI	LLM, Question Classifier, Content Extractor
Tool	HTTP Request, Code Execution, Read Files, JSON Serialize/Deserialize, Text Editor
Logic	Condition Branch, Variable Update, Loop, Parallel Execution
Interaction	User Select, Form Input, Input, Plugin Output
Agent	Agent Node, Sub-Workflow

Key Capabilities

Visual Editor: Drag-and-drop canvas with auto-layout
Node Testing: Test individual nodes in isolation before running the full workflow
Version Management: Save, compare, and restore workflow versions
Run Tracing: Step-by-step execution trace with variable inspection
Loop Support: Nested loop nodes with dedicated inner canvas
Templates: Pre-built templates for common patterns (simple chat, conditional branch)
Auto-save: 5-second debounce with dirty state indicator

📚 Knowledge Base

A complete knowledge management system with AI-powered capabilities:

Documents

Multi-format upload: PDF, DOCX, XLSX, PPTX, images, and more
Chunked upload: Files >2MB automatically split into 2MB chunks (max 500MB)
AI Distillation: Extract key insights from documents using AI
Batch operations: Batch distill, move, and manage documents
Preview: In-app preview for all supported formats
Import/Export: ZIP-based import/export, Obsidian vault import

Notes

Markdown editor: Full-featured editor with wiki-link navigation
Vault system: Create and manage multiple knowledge vaults
Obsidian compatible: Import existing Obsidian vaults

Knowledge Graph

Visual exploration: WebGL-powered graph visualization (PixiJS)
Force-directed layout: Interactive node positioning
Relationship mapping: Discover connections between knowledge nodes

📡 Multi-Channel Support

Connect Octopus to your favorite platforms:

Channel	Features
🖥️ Desktop	Full-featured native app with WebSocket real-time
💬 WeChat	QR code login, send/receive, auto-reply
🐦 Slack	Bolt SDK integration, channel & DM support
🎮 Discord	Bot integration, server & channel messaging
✈️ Telegram	Bot API, chat & group support
📱 DingTalk	Stream protocol, conversation messaging
📧 Email	SMTP/IMAP integration
🔗 Webhook	Generic HTTP webhook for custom integrations
🐦 Feishu	Lark SDK, event subscription

🔌 Extension Ecosystem

Skill Extensions (Just Markdown)

Write a SKILL.md file to teach AI new capabilities:

---
name: "Code Review"
emoji: "🔍"
---

When reviewing code, check for:
1. Security issues (SQL injection, XSS)
2. Performance bottlenecks
3. Naming conventions

Drop it into workspace/extensions/my-skill/SKILL.md and restart to activate.

Extension Marketplace

Browse and install community extensions
Three extension types: Skill, Plugin, Worker
Search, filter by type, sort by popularity
One-click install with environment variable configuration

MCP Protocol Support

Connect to any MCP server (stdio / HTTP SSE)
Auto-discover tools, no manual configuration needed
Visual permission management with enable/disable per tool
Real-time connection status monitoring

🛠️ Built-in Tools

Category	Tools	Description
📁 Filesystem	`read`, `write`, `edit`, `list`	File read/write operations
🖥️ System	`shell`, `spawn`	Command execution
🌐 Network	`web_fetch`	Web content fetching
🖥️ Browser	`browser_navigate`, `browser_click`, `browser_screenshot`, ...	Playwright browser automation
🖼️ Image	`image_understand`, `image_generate`	AI image processing
⏰ Schedule	`cron_add`, `cron_list`, `cron_remove`	Task scheduling
💬 Message	`send_message`	Multi-channel messaging
🧠 Memory	`memory_read`, `memory_write`	Agent memory operations
📚 Knowledge	`knowledge_search`, `knowledge_query`	Knowledge base retrieval
⚡ Action	`action`	Execute extension actions

🧠 Observation & Memory

Octopus automatically extracts insights from conversations:

Observation Types

Type	Description
🎯 Gotcha	Key findings and aha moments
🔧 Problem-Solution	Problem-solution pairs
⚙️ How-it-works	How something works explanations
📝 What-changed	Change records
🔍 Discovery	New discoveries
❓ Why-it-exists	Rationale and reasons
📋 Decision	Design decisions
⚖️ Trade-off	Trade-off analysis
💡 General	General observations

Memory Features

Auto-extraction: AI identifies and extracts observations from conversations
Promote to memory: Elevate important observations to long-term memory
User profiles: Track user preferences and patterns
Contextual depth: View observations with surrounding conversation context

⚙️ Visual Configuration

All configuration has a graphical interface, no YAML required:

Config Item	Description
Model Providers	Add OpenAI/Anthropic/DeepSeek, support multi-provider switching
Agent Settings	Model, max tokens, temperature, max iterations, compression
Channel Config	WeChat QR login, Telegram bot, Slack app, DingTalk, and more
Tool Toggles	Enable/disable tools with one click, set timeout
Workspace	Isolated workspaces with separate config and memory
Budget Limit	Set monthly token limit with over-budget alerts
Multimodal	Image understanding, TTS, and other multimodal settings

💰 Token Usage Visualization

Monitor the cost of every conversation in real-time:

📊 Real-time Stats: Input/output tokens, cache hits, completion tokens, sub-agent usage
📈 Historical Trends: View consumption by day (7/14/30 days)
📋 Breakdown Tables: Per-provider and per-model cost analysis
⚠️ Budget Alerts: Set limits with automatic warnings

⏰ Smart Scheduled Tasks

Not just notifications, but actual work:

SubAgent Execution: Tasks run in isolated agents, performing real operations
Flexible Scheduling: Support ISO time, interval seconds, Cron expressions
Context Inheritance: Tasks can access session memory from creation time
Persistent Storage: Tasks saved in SQLite, survive restarts
Channel Delivery: Send task results to specific channels

🗂️ Workspace Management

Each project has its own isolated workspace:

workspace/
├── project-a/          # Project A
│   ├── extensions/     # Exclusive extensions
│   ├── memory/         # Long-term memory
│   └── history/        # Chat history
├── project-b/          # Project B
│   └── ...

Switch workspace = switch complete config and memory
Export/import workspaces supported
Team sharing: export workspace, colleagues import to use
Built-in file browser with Monaco Editor
Multi-format preview: PDF, DOCX, XLSX, PPTX, images, Markdown

💬 Chat History

All conversations saved in local SQLite
3-level organization: Channel → Session → Instance
Filter by message type, search across history
Return to any historical session anytime
Support parallel multi-sessions

🤖 Visual SubAgent

Create and manage specialized agents through the UI:

Visual Editing: Modify SOUL.md to configure role, tools, model
One-click Creation: Fill in name to auto-generate template config
Isolated Workspace: Each SubAgent has its own config and memory
Master-Slave Dispatch: Main agent automatically calls appropriate SubAgent
Tool & Extension Binding: Assign specific tools and extensions per agent

🚀 Quick Start

Requirements

Node.js >= 18
Python >= 3.10

Install & Run

# 1. Clone repository
git clone <repository-url>
cd octopus

# 2. Install dependencies
npm install

# 3. Start development mode
npm run dev

💡 npm run dev starts both:

Frontend dev server (http://localhost:3000)

Electron desktop window

Python backend (auto-started by Electron)

📦 Build & Release

Development Commands

Command	Description
`npm run dev`	Dev mode (frontend + Electron)
`npm run dev:frontend`	Frontend dev server only
`npm run dev:electron`	Electron only

Build Commands

Command	Description
`npm run build:frontend`	Build React frontend
`npm run build:python`	Package Python backend
`npm run build`	Full build (frontend + Electron)

Package & Release

Command	Description	Output
`npm run dist`	Package current platform	Auto-select by platform
`npm run dist:mac`	macOS package	DMG + ZIP (universal: x64/arm64)
`npm run dist:win`	Windows package	NSIS installer + portable

📂 Output: dist-electron/ 📖 Detailed guide: README_BUILD.md

🏗️ Project Architecture

octopus/
├── agents/                 🧠 AI Agent workspace
│   ├── code-reviewer/      Code review agent
│   ├── common/             Common agent templates
│   └── system/             System agent config
│       └── avatars/        Agent avatar assets
├── backend/                ⚡ Python backend
│   ├── agent/              Agent core logic
│   │   ├── processors/     Streaming / non-streaming / longtask processors
│   │   ├── compressor.py   Context compression
│   │   ├── subagent.py     SubAgent dispatch
│   │   └── observation_*.py Observation extraction & management
│   ├── api/                FastAPI service interface
│   ├── channels/           Multi-channel support
│   │   ├── desktop/        Desktop channel (WebSocket)
│   │   ├── wechat/         WeChat channel
│   │   ├── feishu/         Feishu/Lark channel
│   │   ├── dingtalk/       DingTalk channel
│   │   ├── slack/          Slack channel
│   │   ├── discord/        Discord channel
│   │   ├── telegram/       Telegram channel
│   │   ├── email/          Email channel
│   │   └── webhook/        Webhook channel
│   ├── core/               Core modules
│   │   ├── config/         Configuration & schema
│   │   ├── events/         Event bus system
│   │   ├── longtask/       Long-running task management
│   │   ├── models/         Data models
│   │   └── providers/      LLM provider adapters (OpenAI/Anthropic)
│   ├── data/               Data storage (SQLite)
│   │   ├── migrations/     Database migrations (11 migrations)
│   │   └── schema/         Data schemas (agent/session/token/workflow/...)
│   ├── extensions/         Plugin system
│   │   ├── builtin/        Built-in extensions (cron, etc.)
│   │   └── loader.py       Dynamic extension loader
│   ├── mcp/                MCP protocol integration
│   │   ├── server/         MCP server connection & tool registry
│   │   └── llm_bridge.py   LLM-MCP bridge
│   ├── services/           Service layer
│   │   ├── cron/           Scheduled task service
│   │   ├── tts/            Text-to-speech (OpenAI/MiMo engines)
│   │   ├── workflow/       Workflow engine & executor
│   │   ├── knowledge_*.py  Knowledge base services
│   │   ├── image_service.py Image generation service
│   │   └── llm_service.py  LLM invocation service
│   ├── tools/              Built-in tools
│   │   ├── filesystem.py   Filesystem tools
│   │   ├── shell.py        Shell tools
│   │   ├── web_fetch.py    Web fetch tools
│   │   ├── browser/        Playwright browser automation
│   │   ├── image.py        Image processing tools
│   │   ├── cron.py         Cron task tools
│   │   ├── message.py      Message tools
│   │   ├── memory.py       Memory read tools
│   │   ├── memory_write.py Memory write tools
│   │   ├── knowledge.py    Knowledge base tools
│   │   ├── action.py       Extension action tools
│   │   └── spawn.py        Process spawn tools
│   └── utils/              Utility functions
├── electron/               🖥️ Electron main process
│   ├── main.js             Main entry (Python lifecycle, window management)
│   └── preload.js          Preload script (IPC bridge)
├── frontend/               🎨 React frontend
│   ├── src/
│   │   ├── pages/          Page components
│   │   │   ├── Chat/       Chat interface with streaming & tool display
│   │   │   ├── Config/     Settings (providers/agent/channels/multimodal)
│   │   │   ├── Workflow/   Visual workflow editor (ReactFlow)
│   │   │   ├── Knowledge/  Knowledge base (documents/notes/graph)
│   │   │   ├── Agents/     SubAgent management
│   │   │   ├── MCP/        MCP server & tool management
│   │   │   ├── Extensions/ Extension marketplace
│   │   │   ├── Cron/       Scheduled tasks
│   │   │   ├── Tokens/     Token usage dashboard
│   │   │   ├── History/    Chat history browser
│   │   │   ├── Memory/     Observation & memory viewer
│   │   │   └── Workspace/  File browser & editor
│   │   ├── components/     Shared components
│   │   │   ├── MessageList/ Message rendering with iteration folds
│   │   │   ├── TTSPlayer/  Audio playback
│   │   │   ├── TaskIndicator/ Task status indicator
│   │   │   └── MermaidDiagram/ Mermaid chart rendering
│   │   ├── workflow/       Workflow engine
│   │   │   ├── components/ Node components (13 registered types)
│   │   │   ├── hooks/      Zustand workflow store
│   │   │   ├── types/      Type definitions
│   │   │   └── templates/  Workflow templates
│   │   ├── contexts/       React contexts (WebSocket, DistillTask)
│   │   ├── hooks/          Custom hooks (useChatState, useMermaid)
│   │   └── utils/          Utilities
│   └── package.json
├── build/                  🔧 Build resources (icons, etc.)
├── workspace/              📂 Workspace data (runtime-generated, git-ignored)
├── build_python.py         🐍 Python packaging script (PyInstaller)
├── package.json            📋 Project config & scripts
└── README.md               📖 Project documentation

Tech Stack

Layer	Technology	Description
Frontend	React 18 + Vite 5	Modern UI framework
	Ant Design 6	Component library
	ReactFlow	Visual workflow editor
	Monaco Editor	Code editor
	ECharts 6	Data visualization
	PixiJS	Knowledge graph WebGL rendering
	Zustand	Workflow state management
Backend	Python 3.10+ + FastAPI	High-performance async web service
	SQLite + SQLAlchemy	Local lightweight database
	Playwright	Browser automation
	APScheduler	Task scheduling
Desktop	Electron 28	Cross-platform desktop framework
	electron-builder	App packaging tool

Runtime Architecture

┌─────────────────────────────────────────────┐
│                Electron Main Process         │
│  ┌──────────────────┐  ┌─────────────────┐  │
│  │  BrowserWindow   │  │ Python Process   │  │
│  │  (React SPA)     │  │ (octopus-server) │  │
│  │                  │  │                  │  │
│  │  electronAPI ────┼──┼──▶ FastAPI       │  │
│  │  (preload bridge)│  │    (WebSocket)   │  │
│  └──────────────────┘  └─────────────────┘  │
└─────────────────────────────────────────────┘

Communication: Full WebSocket between frontend and backend
Request-Response: request_id based correlation with timeout
Event Subscription: Pub/Sub pattern for real-time events
Python Lifecycle: Managed by Electron (auto-start/stop)

🔧 Model Configuration

Add API keys in the app settings panel:

Supported Providers

Provider	Representative Models
OpenAI	GPT-4o, GPT-4 Turbo, GPT-3.5 Turbo, o1
Anthropic	Claude 3 Opus, Claude 3 Sonnet, Claude 3 Haiku
Google	Gemini Pro, Gemini Ultra
DeepSeek	DeepSeek Chat, DeepSeek Coder
Alibaba	Tongyi Qianwen series
Baidu	Wenxin Yiyan series
Custom	Any OpenAI-compatible API endpoint

Configuration Steps

Open app → Settings → Model Providers
Add provider (select or custom)
Enter API Key & Base URL
Select model to use
Save and start

🔌 MCP Protocol

Octopus fully supports Model Context Protocol (MCP):

🔗 Connect to any MCP server
🛠️ Use tools provided by MCP
🔐 Secure permission management
🔄 Real-time connection monitoring
📋 Visual server management (add/edit/delete/reconnect)
🔍 Auto-discovered tools with per-tool enable/disable

Supported Transports

stdio: Local process communication
HTTP SSE: Server-Sent Events over HTTP

🤖 Agent Workspace

Agent system supports continuous memory and personalization:

Configuration Files

File	Purpose
`SOUL.md`	Agent soul - core principles and personality
`IDENTITY.md`	Agent identity - self-introduction
`AGENTS.md`	Workspace guide - usage instructions
`MEMORY.md`	Long-term memory - important info persistence
`memory/YYYY-MM-DD.md`	Daily notes - daily event records

Creating Custom Agents

Create new folder in agents/ directory, add config files to create custom agent.

📂 Project Structure

octopus/
├── backend/              # Python backend (FastAPI)
├── frontend/             # React frontend (Vite)
├── electron/             # Electron main process
├── build/                # Build resources (icons, etc.)
├── build_python.py       # Python packaging script
├── workspace/            # ⚠️ Runtime-generated directory (git-ignored)
│   ├── agents/           #   - User-created agent configurations
│   ├── extensions/       #   - Installed extensions
│   ├── files/            #   - Workspace files
│   ├── images/           #   - Generated images
│   └── ...               #   - Other runtime data
└── scripts/              # Helper scripts

Note: The workspace/ directory is created at runtime and contains user data, agent configs, and generated files. It's excluded from version control by .gitignore.

📖 Documentation

📘 Build Guide - Packaging & release details
📗 Agent Guide - Agent workspace usage
📕 Identity - Learn who Octopus is
🧠 Soul Core - Agent core principles
🔌 MCP Docs - MCP protocol integration
🌐 Browser Tools - Browser automation guide

🤝 Contributing

Issues and Pull Requests welcome:

🐛 Bug reports
✨ New features
📝 Documentation improvements
🎨 UI/UX optimizations

📋 Changelog

2026-05

Date	Version	Changes
2026-05-17	v1.0.0	🔄 New: Visual Workflow editor with 24 node types
2026-05-17	v1.0.0	📚 New: Knowledge Base with documents, notes, graph
2026-05-17	v1.0.0	📡 New: Multi-channel support (Slack/Discord/Telegram/...)
2026-05-17	v1.0.0	🌐 New: Playwright browser automation tools
2026-05-17	v1.0.0	🧠 New: Observation & memory system

2026-03

Date	Version	Changes
2026-03-29	v1.0.0	🔊 New: Text-to-Speech (TTS) feature support
2026-03-29	v1.0.0	🤖 New: SubAgent management and UI improvements
2026-03-28	v1.0.0	🗜️ New: Context compression and LLM retry optimization
2026-03-25	v1.0.0	📄 New: PDF, DOCX, and Excel file support
2026-03-24	v1.0.0	💬 New: WeChat channel with QR login and messaging
2026-03-22	v1.0.0	🖼️ New: Frameless window support
2026-03-20	v1.0.0	🎉 Release: Project renamed to Octopus

🐙 Octopus makes your work more efficient 🐙

_{Built with ❤️ and 🐙 tentacles}