MetaAgent: Toward Self-Evolving Agent via Tool Meta-Learning

Infrastructure | Quick-Start | FAQs

Overview

MetaAgent is a next-generation agentic AI framework built on the principle of learning-by-doing: expertise is developed through hands-on practice and continual self-improvement, not through static rules or costly retraining. MetaAgent begins with a minimal workflow, equipped only with essential reasoning and adaptive help-seeking capabilities. When encountering knowledge gaps, it generates natural language help requests, which are flexibly routed to external tools by a dedicated tool router.

As MetaAgent solves tasks, it performs ongoing self-reflection and answer verification, turning experience into concise, generalizable lessons that are dynamically integrated into future tasks. Over time, MetaAgent autonomously builds in-house tools and a persistent knowledge base by organizing its tool-use history—enabling ever more efficient information retrieval and integration. We call this continual, data-driven improvement process meta tool learning. Unlike traditional agentic systems, MetaAgent evolves and adapts on-the-fly—without changing model weights or needing additional training.

:sparkles: Features

• Minimal, General Workflow: Starts with core reasoning and help-seeking abilities, adaptable to diverse tasks.
• Flexible Tool Routing: Dedicated router maps natural language requests to appropriate external tools.
• Meta Tool Learning: Uses self-reflection and answer verification to improve reasoning and tool use.
• Dynamic Context Engineering: Distills experience into lessons that update future inputs without retraining.
• Persistent Knowledge Base: Builds and updates internal memory from tool interactions for better retrieval.

:rocket: Infrastructure

Efficient infrastructure is crucial for deep search applications, as it helps save resources and improve overall performance. Here, we provide a service that combines Google Search and Jina for web page retrieval. This service exposes an HTTP API that accepts two parameters: query and topk. It first uses the Google Search API to obtain search results, then leverages the Jina Reader API to fetch the content of the resulting web pages.

To optimize resource usage, we deploy a MySQL service to cache both search results and web page contents. This ensures that repeated queries are served quickly without redundant external requests.

Setup Instructions

1. Deploy MySQL with Docker

   If this is your first time using the service, pull the MySQL Docker image:

   docker pull mysql:8.0
   

2. Start MySQL Container with Custom User, Password, and Port

   Run the MySQL container with your own username, password, and port (e.g., 3306):

   docker run -d \
     --name metaagent-mysql \
     -e MYSQL_ROOT_PASSWORD=your_password \
     -e MYSQL_USER=your_user \
     -e MYSQL_PASSWORD=your_password \
     -e MYSQL_DATABASE=search \
     -p 3306:3306 \
     mysql:8.0
   

   Replace your_user and your_password with your desired username and password. You can also change -p 3306:3306 to use a different port if needed.

   After starting, the MySQL service will be available on your local machine at port 3306.

   Check the script at tools/db/build_db.py and use the build_db(db_name) function to create the necessary database structure.

3. Configure API Keys

   Update the configuration file at data/api_dict.json with your Google Search API key, CSE ID, and Jina API key. Example:

   {
       "db": {
           "root": {
               "host": "localhost",
               "port": 3306,
               "user": "your_user",
               "password": "your_password"
           }
       },
       "jina": {
           "api_key": "your_jina_api_key"
       },
       "search_engine": {
           "google": {
               "api_key": "your_google_api_key",
               "cse_id": "your_cse_id"
           }
       }
   }
   

4. Run the Search Service

   Start the search and web scraping service locally:

   python -m tools.db.search_app
   

   You can configure the port and other settings in the tools/db/search_app.py file. If needed, you can also replace this service with your own custom search backend.

This setup provides a robust, resource-efficient search infrastructure for deep search tasks, combining fast retrieval with persistent caching.

A case:

curl localhost:12347/search_v1?query=google&topk=5

You will get the following response:

{
  "query": "google",
  "total_results": 5,
  "results": [
    {
      "id": 1,
      "title": "Google",
      "url": "https://www.google.com/",
      "site_name": "www.google.com",
      "date": "",
      "snippet": "Search the world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking ...",
      "context": "..."
    },
  ]
}

Along with the web search service, we highly recommend organizing your search cache in a local data warehouse. In this project, we store cached web pages in Elasticsearch (ES). As your ES cache grows, you can perform searches not only via web search but also directly over your local cache, which can significantly improve both efficiency and search quality.

Setting Up Elasticsearch

1. Download and Start Elasticsearch

   Download Elasticsearch from the official website. After extracting, start the ES service:

   ./bin/elasticsearch
   

2. Build and Populate the ES Index

   Check the script at tools/es/build_index.py. On first use, you should run the create_index() function to initialize the index. Afterwards, you can index cached web pages into ES as they are collected.

3. Hybrid Search with Embeddings

   We use hybrid search in this project, combining traditional keyword search with dense retrieval using embeddings. For embedding, we use the bge-m3 model. You need to start a vLLM server with this model:

   vllm serve ... --port 25883
   

   (Replace ... with the appropriate model path and options.)

4. Start the Local ES Search Service

   Launch the cache-based search service:

   python -m tools.es.cache_search_app
   

   This will provide a search API powered by your local Elasticsearch instance, leveraging both keyword and embedding-based retrieval for improved results.

By combining web search with a robust local cache in Elasticsearch and hybrid search techniques, you can achieve faster and more accurate search results for deep search tasks.

A case:

curl localhost:12348/search -X POST -H "Content-Type: application/json" -d '{"query": "nature", "topk": 5}'

You will get the following response:

{
  "results": [
    {
      "content": "Published Time: Wed, 04 Jun 2025 12:01:25 GMT\n\n# WRITING NATURE",
      "score": 1.5914185,
      "snippet": "the poem you wanted, so I hope you will forgive me for sending it to ... ThE PoET's PAEAN To ThE WATER cyclE, TURNEd To. A PhoToGRAPhER's cElEBRATIoN ...",
      "title": "WRITING NATURE",
      "url": "https://aba.org.uk/assets/catalogues/215_DPS.pdf"
    }
  ],
}

In addition to the /search route, the tools/es/cache_search_app.py service also provides an /insert route. This allows you to add new webpages to the Elasticsearch index by sending a POST request with the webpage data.

Besides, you can also manually call add_webpages_to_index() function in tools/es/build_index.py to add new webpages to the Elasticsearch index.

Tip: The infrastructure described here is just one possible implementation used in my own project, mainly because deep search tasks require frequent use of these tools and rely heavily on web search. If you have a better solution or a different setup that fits your needs, feel free to replace or modify this part. This infrastructure is entirely optional and not required for all use cases.

:notebook: Quick Start

To quickly evaluate MetaAgent, please refer to the technical report for detailed instructions. You can run the evaluation with the following command:

python src/run_evaluation.py \
    --reasoning_model QwQ-32B \
    --reasoning_model_base_url http://localhost:12345/v1/ \
    --reasoning_model_api_key empty \
    --auxiliary_model Qwen2.5-7B-Instruct \
    --auxiliary_model_base_url http://localhost:12346/v1/ \
    --auxiliary_model_api_key empty \
    --search_api_url http://localhost:12347/search \
    --cache_search_url http://localhost:12348/search \
    --max_retries 3 \
    --search_topk 10 \
    --use_experience \
    --use_web_search \
    --use_cache_search \
    --use_llm_equivalence \
    --eval_task GAIA \
    --version v1 \
    # --advanced_reasoning_model google/gemini-2.5-flash \
    # --advanced_reasoning_model_base_url https://openrouter.ai/api/v1 \
    # --advanced_reasoning_model_api_key openrouter-api-key

Make sure you have the following services running:

• Qwen2.5-7B-Instruct: http://localhost:12346/v1/
• QwQ-32B: http://localhost:12345/v1/
• Search Service: http://localhost:12347/search_v1
• Cache Search Service: http://localhost:12348/search
• MySQL: http://localhost:3306
• Elasticsearch: http://localhost:9200

The Qwen models are hosted using VLLM:

vllm serve qwen2.5-7b-instruct --port 12346
vllm serve QwQ-32B --port 12345

You can use other OpenAI-style models by replacing the --reasoning_model and --auxiliary_model arguments with the model name. Besides, if advanced reasoning model is specified, MetaAgent will use the advanced reasoning model as the central reasoning model.

Tip: Check all ports in these services to enable communication between the services.

In the data folder, we provide the GAIA subset and the WebWalker dataset used in this project. The BrowseComp dataset, also utilized here, is not publicly released in decoded form by the original authors; you will need to download and decode it yourself following their instructions.

:chart_with_upwards_trend: Evaluation

We evaluate MetaAgent on the GAIA, WebWalkerQA, and BrowseComp datasets. The results are shown in the following table:

Method	General AI Assistant				WebWalkerQA				BrowseComp
	Level 1	Level 2	Level 3	Avg.	Easy	Medium	Hard	Avg.	Art	History	Avg.
Direct Reasoning (w/o Retrieval)
Qwen2.5-32B	20.5	9.6	8.3	13.6	3.8	2.5	3.3	3.1	0.0	0.0	0.0
QwQ-32B	30.8	15.4	25.0	22.3	7.5	2.1	4.6	4.3	0.0	0.0	0.0
GPT-4o	23.1	15.4	8.3	17.5	6.7	6.0	4.2	5.5	0.8	0.8	0.8
DeepSeek-R1-671B	43.6	26.9	8.3	31.1	5.0	11.8	11.3	10.0	0.0	0.0	0.0
Direct Reasoning (w/ Retrieval)
RAG (Qwen2.5-32B)	12.8	11.8	8.3	11.8	23.1	14.3	11.3	15.3	0.0	0.0	0.0
RAG (QwQ-32B)	33.3	36.5	8.3	32.0	36.9	26.1	33.5	31.2	0.0	0.0	0.0
Workflow-Based Agent
Query Planning (Qwen2.5-32B)	30.8	17.3	0.0	20.4	29.4	36.4	25.0	30.7	0.0	0.0	0.0
Query Planning (QwQ-32B)	48.7	25.0	8.3	32.0	28.8	35.7	30.8	32.5	0.0	0.8	0.4
Iterative RAG (Qwen2.5-32B)	35.9	19.2	8.3	24.3	30.6	35.7	25.4	30.9	0.0	0.0	0.0
Iterative RAG (QwQ-32B)	51.3	28.8	8.3	35.0	29.4	32.9	31.3	31.5	0.8	0.0	0.4
ReAct (Qwen2.5-32B)	46.1	44.2	8.3	40.7	44.3	46.7	29.2	38.4	0.0	0.0	0.0
ReAct (QwQ-32B)	48.7	34.6	16.7	37.8	35.6	29.1	13.2	24.1	0.8	0.8	0.8
ReAct (GPT-4o)	51.2	34.6	8.3	34.6	34.6	42.0	23.9	33.8	2.4	1.6	1.9
Search-o1-32B	53.8	44.2	16.7	39.8	43.1	35.0	27.1	34.1	1.6	2.4	1.9
End-to-end optimized Agent
WebThinker-32B-Base	53.8	44.2	16.7	44.7	47.5	41.1	39.2	41.9	2.4	2.4	2.3
WebThinker-32B-RL	56.4	50.0	16.7	48.5	58.8	44.6	40.4	46.5	2.4	3.1	2.7
Our method
MetaAgent (QwQ-32B)	61.5	42.3	25.0	47.6	55.0	49.6	47.9	52.1	7.9	6.4	7.1

:raised_hands: FAQs

If you have any questions, please feel free to contact me at tommy[at]chien.io.

Citation:

@misc{qian2025metaagentselfevolvingagenttool,
      title={MetaAgent: Toward Self-Evolving Agent via Tool Meta-Learning}, 
      author={Hongjin Qian and Zheng Liu},
      year={2025},
      eprint={2508.00271},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2508.00271}, 
}
}