MCP IoT Control Usage

This document describes how to implement IoT control for ESP32 devices using the MCP protocol. For the detailed wire protocol, see mcp-protocol.md.

Introduction

MCP (Model Context Protocol) is the recommended protocol for IoT control in this project. It uses JSON-RPC 2.0 to let the backend discover and invoke "tools" registered by the device, giving you a flexible way to expose device functionality.

Typical Flow

1. The device boots and connects to the backend over WebSocket or MQTT.
2. The backend sends an initialize call to start the MCP session.
3. The backend issues tools/list to discover available tools and their input schemas.
4. The backend calls individual tools with tools/call to control the device.

See mcp-protocol.md for the exact message format.

Registering Tools on the Device

Tools are registered through the McpServer singleton. There are two registration APIs:

• McpServer::AddTool - regular tool, visible in the default tools/list response and callable by the AI model.
• McpServer::AddUserOnlyTool - hidden tool, only returned when the backend lists tools with withUserTools=true. Use this for privileged or user-initiated actions (reboot, firmware upgrade, snapshots, etc.) that must not be invoked autonomously by the model.

Both APIs share the same signature:

void AddTool(
    const std::string& name,           // unique tool name, e.g. self.dog.forward
    const std::string& description,    // short description for the model
    const PropertyList& properties,    // input parameters (may be empty); supported types: bool, int, string
    std::function<ReturnValue(const PropertyList&)> callback // implementation
);

void AddUserOnlyTool(
    const std::string& name,
    const std::string& description,
    const PropertyList& properties,
    std::function<ReturnValue(const PropertyList&)> callback
);

• name - unique identifier. A module.action naming style works well.
• description - natural-language description; used by the AI to decide when to call the tool.
• properties - input parameters. Supported property types are boolean, integer, and string, with optional min/max and default values.
• callback - implementation. Return values may be bool, int, or std::string.

Example (ESP-Hi)

void InitializeTools() {
    auto& mcp_server = McpServer::GetInstance();

    // Example 1: no arguments - move the robot forward
    mcp_server.AddTool("self.dog.forward",
        "Move the robot forward",
        PropertyList(),
        [this](const PropertyList&) -> ReturnValue {
            servo_dog_ctrl_send(DOG_STATE_FORWARD, NULL);
            return true;
        });

    // Example 2: with arguments - set RGB light color
    mcp_server.AddTool("self.light.set_rgb",
        "Set the RGB color of the light",
        PropertyList({
            Property("r", kPropertyTypeInteger, 0, 255),
            Property("g", kPropertyTypeInteger, 0, 255),
            Property("b", kPropertyTypeInteger, 0, 255)
        }),
        [this](const PropertyList& properties) -> ReturnValue {
            int r = properties["r"].value<int>();
            int g = properties["g"].value<int>();
            int b = properties["b"].value<int>();
            led_on_ = true;
            SetLedColor(r, g, b);
            return true;
        });
}

Example - Registering a User-only Tool

mcp_server.AddUserOnlyTool("self.display.clear_cache",
    "Clear locally cached images. User-only action.",
    PropertyList(),
    [](const PropertyList&) -> ReturnValue {
        ClearLocalCache();
        return true;
    });

A tool registered this way will not appear in a regular tools/list response. The backend must set params.withUserTools = true to see it.

Built-in Tools

McpServer::AddCommonTools and McpServer::AddUserOnlyTools register a number of tools automatically:

Default (AI-callable) tools - from AddCommonTools

Tool	Description
self.get_device_status	Returns the current volume, screen, battery, network, etc.
self.audio_speaker.set_volume	Set speaker volume (volume: 0-100).
self.screen.set_brightness	Set screen brightness when a backlight is available (brightness: 0-100).
self.screen.set_theme	Switch UI theme (theme: "light" or "dark"), when LVGL is enabled.
self.camera.take_photo	Take a picture with the on-board camera (when the board has one) and answer the given question about it.

Board-specific tools are appended after these by each board's InitializeTools().

User-only tools - from AddUserOnlyTools

These tools are hidden by default. The backend must pass withUserTools=true to tools/list to see them. They are intended for companion apps / end users rather than the AI model.

Tool	Description
self.get_system_info	Return a JSON blob describing the system.
self.reboot	Reboot the device after a short delay.
self.upgrade_firmware	Download firmware from url and install it, then reboot.
self.screen.get_info	Return the current screen width, height, and whether it is monochrome (LVGL boards only).
self.screen.snapshot	Snapshot the screen as JPEG and upload it to url (LVGL boards, when CONFIG_LV_USE_SNAPSHOT=y).
self.screen.preview_image	Download and display an image from url on the screen.
self.assets.set_download_url	Set the download URL for the assets partition.

JSON-RPC Examples

1. Get the tools list

{
  "jsonrpc": "2.0",
  "method": "tools/list",
  "params": { "cursor": "", "withUserTools": false },
  "id": 1
}

2. Move the chassis forward

{
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "name": "self.chassis.go_forward",
    "arguments": {}
  },
  "id": 2
}

3. Switch the light mode

{
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "name": "self.chassis.switch_light_mode",
    "arguments": { "light_mode": 3 }
  },
  "id": 3
}

4. Reboot the device (user-only)

{
  "jsonrpc": "2.0",
  "method": "tools/call",
  "params": {
    "name": "self.reboot",
    "arguments": {}
  },
  "id": 4
}

Notes

• Tool names, parameters, and return values must match what the device registers via AddTool / AddUserOnlyTool.
• Prefer MCP for any new IoT control.
• For the wire protocol and advanced topics, see mcp-protocol.md.