Introduction

October 6, 2025 ยท View on GitHub

DecEx-RAG: Boosting Agentic Retrieval-Augmented Generation with Decision and Execution Optimization via Process Supervision

Introduction

Agentic Retrieval-Augmented Generation (Agentic RAG) enhances the processing capability for complex tasks through dynamic retrieval and adaptive workflows. Recent advances (e.g., Search-R1) have shown that outcome-supervised reinforcement learning demonstrate strong performance. However, this approach still suffers from inefficient exploration, sparse reward signals, and ambiguous global reward feedback. To address these challenges, we propose DecEx-RAG, which models RAG as a Markov Decision Process (MDP) incorporating decision-making and execution, while introducing an efficient pruning strategy to optimize data expansion. Through comprehensive process-level policy optimization, DecEx-RAG significantly enhances the autonomous task decomposition, dynamic retrieval, and high-quality answer generation capabilities of large language models (LLMs). Experiments show that DecEx-RAG achieves an average absolute performance improvement of 6.2% across six datasets, significantly outperforming existing baselines. Moreover, the pruning strategy improves data construction efficiency by nearly 6x, providing an efficient solution for process-supervised RAG training.

Overview

Figure 1: Illustration of the framework for DecEx-RAG, demonstrates the process of search tree expansion and pruning.

Main Results

Table 1: The overall experimental results of DecEx-RAG and other baselines on six datasets. The best/second best scores in each dataset are bolded/underlined.

Installation

DecEx-RAG environment

pip install torch==2.6.0
pip install vllm==0.8.5.post1
pip install trl

Retriever environment (From Search-R1)

If you would like to call a local retriever as the search engine, you can install the environment as follows. (We recommend using a seperate environment.)

conda create -n retriever python=3.10
conda activate retriever

# we recommend installing torch with conda for faiss-gpu
conda install pytorch==2.4.0 torchvision==0.19.0 torchaudio==2.4.0 pytorch-cuda=12.1 -c pytorch -c nvidia
pip install transformers datasets pyserini

## install the gpu version faiss to guarantee efficient RL rollout
conda install -c pytorch -c nvidia faiss-gpu=1.8.0

## API function
pip install uvicorn fastapi

Quick Start

Corpus and Index

We followed the experimental setup of Search-R1, and the corpus and index can be downloaded through their repository.

Data Generation

1. start the retrieval service

bash run_retrieval.sh

2. rollout

bash rollout.sh

Training

# stage 1: SFT
bash sft_train.sh
# stage 2: DPO
bash dpo_train.sh

Evaluation

1. start the retrieval service

bash run_retrieval.sh

2. evaluation

bash eval.sh