Developer Guide

July 28, 2026 · View on GitHub

中文 | English

Developer Guide

Welcome to participate in the development of the DataAgent project! This document will help you understand how to contribute to the project.

Development Environment Setup

Prerequisites

  • JDK: 17 or higher
  • Maven: 3.6 or higher
  • Node.js: 22
  • pnpm: 11
  • MySQL: 5.7 or higher
  • Docker: Required only when running or verifying Python workflows
  • Git: Version control tool
  • IDE: IntelliJ IDEA or Eclipse (IntelliJ IDEA recommended)

Clone Project

git clone https://github.com/spring-ai-alibaba/DataAgent.git
cd DataAgent

Backend Development Environment

  1. Import Project into IDE

    • Open the project root directory with IntelliJ IDEA
    • IDE will automatically recognize it as a Maven project and download dependencies

  2. Configure Database

    • Create a MySQL database
    • Modify the database configuration in data-agent-management/src/main/resources/application.yml

  3. Start Backend Service

    ./mvnw -pl data-agent-management spring-boot:run

Frontend Development Environment

  1. Install Dependencies

    cd data-agent-frontend-nuxt
pnpm install

  2. Start Development Server

    pnpm dev

  3. Access Application

Core Module Description

1. StateGraph Workflow Engine

The workflow is based on Spring AI Alibaba's StateGraph implementation. Core nodes include:

  • IntentRecognitionNode: Intent recognition
  • EvidenceRecallNode: Evidence recall
  • PlannerNode: Plan generation
  • SqlGenerateNode: SQL generation
  • PythonGenerateNode: Python code generation
  • PythonExecuteNode: Parse PEP 723 metadata and dispatch SAA sandbox execution
  • PythonAnalyzeNode: Analyze Python output and update step state
  • ReportGeneratorNode: Report generation

2. Multi-Model Scheduling

Multi-model management and hot-swapping is implemented through AiModelRegistry:

@Service
public class AiModelRegistry {
    private ChatModel currentChatModel;
    private EmbeddingModel currentEmbeddingModel;

    public void refreshChatModel(ModelConfig config) {
        // Dynamically create and switch Chat model
    }

    public void refreshEmbeddingModel(ModelConfig config) {
        // Dynamically create and switch Embedding model
    }
}

3. Vector Retrieval Service

AgentVectorStoreService provides a unified vector retrieval interface:

@Service
public class AgentVectorStoreService {
    public List<Document> retrieve(String query,
                                   String agentId,
                                   VectorType vectorType) {
        // Vector retrieval logic
    }
}

Coding Standards

Java Coding Standards

  1. Naming Conventions

    • Class names: PascalCase
    • Method names: camelCase
    • Constants: UPPER_SNAKE_CASE

  2. Comment Standards

    • All public classes and methods must have JavaDoc comments
    • Complex logic requires inline comments

  3. Code Format

    • Use 4 spaces for indentation
    • Each line of code should not exceed 120 characters
    • Use Google Java Style Guide

TypeScript Coding Standards

  1. Naming Conventions

    • Component names: PascalCase
    • Variables/functions: camelCase
    • Interfaces: I prefix + PascalCase

  2. Type Definitions

    • Prefer interface over type
    • Avoid using any type
    • Add types for all function parameters and return values

  3. Code Format

    • Use 2 spaces for indentation
    • Use Prettier for code formatting
    • Use ESLint for code quality checking

Development Configuration Manual

All configuration items in this project are under the spring.ai.alibaba.data-agent prefix.

1. General Configuration

Configuration ItemDescriptionDefault Value
spring.ai.alibaba.data-agent.llm-service-typeLLM service type (STREAM/BLOCK)STREAM
spring.ai.alibaba.data-agent.max-sql-retry-countSQL execution failure retry count10
spring.ai.alibaba.data-agent.max-sql-optimize-countMaximum SQL optimization attempts10
spring.ai.alibaba.data-agent.sql-score-thresholdSQL optimization score threshold0.95
spring.ai.alibaba.data-agent.maxturnhistoryMaximum conversation turns to retain5
spring.ai.alibaba.data-agent.maxplanlengthMaximum plan length limit per planning2000
spring.ai.alibaba.data-agent.max-columns-per-tableMaximum estimated columns per table50
spring.ai.alibaba.data-agent.fusion-strategyMulti-channel recall result fusion strategyrrf
spring.ai.alibaba.data-agent.enable-sql-result-chartEnable SQL result chart judgmenttrue
spring.ai.alibaba.data-agent.enrich-sql-result-timeoutSQL result chart generation timeout (ms)3000

2. Embedding Batch Configuration

Configuration prefix: spring.ai.alibaba.data-agent.embedding-batch

Configuration ItemDescriptionDefault Value
encoding-typeText encoding type (refer to com.knuddels.jtokkit.api.EncodingType)cl100k_base
max-token-countMaximum tokens per batch. Recommended: 2000-80008000
reserve-percentageReserve percentage (for buffer space)0.2
max-text-countMaximum texts per batch (DashScope limit is 10)10

3. Vector Store Configuration

Configuration prefix: spring.ai.alibaba.data-agent.vector-store

Configuration ItemDescriptionDefault Value
default-similarity-thresholdGlobal default similarity threshold0.4
table-similarity-thresholdTable recall similarity threshold0.2
batch-del-topk-limitMaximum documents for batch deletion5000
default-topk-limitGlobal default max documents returned (currently only used by business knowledge and agent knowledge)8
table-topk-limitMaximum documents for table recall10
enable-hybrid-searchEnable hybrid searchfalse
elasticsearch-min-scoreES keyword search minimum score threshold0.5

Vector Store Dependency Extension

The project uses in-memory vector store (SimpleVectorStore) by default. To use persistent vector stores (like PGVector, Milvus, etc.), follow these steps:

  1. Add Dependency: Add the corresponding Spring AI Starter to pom.xml.

    <!-- Example: Import PGvector -->
<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-vector-store-pgvector</artifactId>
</dependency>

  2. Configure Properties: Add the corresponding vector store connection configuration in application.yml. For specific parameters, refer to Spring AI Official Documentation.

  3. Configure spring.ai.vectorstore.type. You can find the specific value after importing the vector store starter above by searching for VectorStoreAutoConfiguration auto-configuration class. For example, for es it's ElasticsearchVectorStoreAutoConfiguration, and you can see that spring.ai.vectorstore.type expects elasticsearch.

ES Schema Configuration Example

Below is the Elasticsearch Schema structure. Other vector stores (like Milvus, PGVector) can reference this structure to create their Schema, paying special attention to the data types of fields in metadata.

{
  "mappings": {
    "properties": {
      "content": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "embedding": {
        "type": "dense_vector",
        "dims": 1024,
        "index": true,
        "similarity": "cosine",
        "index_options": {
          "type": "int8_hnsw",
          "m": 16,
          "ef_construction": 100
        }
      },
      "id": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword",
            "ignore_above": 256
          }
        }
      },
      "metadata": {
        "properties": {
          "agentId": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "agentKnowledgeId": {
            "type": "long"
          },
          "businessTermId": {
            "type": "long"
          },
          "concreteAgentKnowledgeType": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          },
          "vectorType": {
            "type": "text",
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
        }
      }
    }
  }
}

4. Text Splitter Configuration

Configuration prefix: spring.ai.alibaba.data-agent.text-splitter

Configuration ItemDescriptionDefault Value
chunk-sizeDefault chunk size (token-based)1000
min-chunk-size-charsMinimum chunk character count400
min-chunk-length-to-embedMinimum chunk length for embedding10
max-num-chunksMaximum number of chunks5000
keep-separatorKeep separatortrue
separatorsCustom separator listnull (use default)

5. Code Executor Configuration

Configuration prefix: spring.ai.alibaba.data-agent.code-executor

Configuration ItemDescriptionDefault Value
code-timeoutPython code execution timeout60s
limit-memoryContainer memory limit (MB)500
cpu-coreContainer CPU cores1
python-max-tries-countMaximum Python execution retries5
sandbox.docker-hostDocker Engine endpointunix:///var/run/docker.sock
sandbox.image-nameSAA base imageagentscope-registry.ap-southeast-1.cr.aliyuncs.com/agentscope/runtime-sandbox-base:latest
sandbox.container-prefixTask container name prefixdataagent-sandbox-
sandbox.max-concurrencyMaximum concurrent sandboxes4
sandbox.queue-capacityBounded wait queue size10
sandbox.max-code-bytesPython source UTF-8 byte limit262144 (256 KiB)
sandbox.max-input-bytesstdin JSON UTF-8 byte limit10485760 (10 MiB)
sandbox.max-output-bytesstdout UTF-8 byte limit1048576 (1 MiB)
sandbox.max-error-bytesstderr UTF-8 byte limit262144 (256 KiB)
sandbox.max-metadata-bytesPEP 723 metadata UTF-8 byte limit8192 (8 KiB)
sandbox.max-dependenciesMaximum number of direct dependencies20
sandbox.package-index-urlDynamic package indexhttps://pypi.org/simple
sandbox.dependency-install-timeoutDependency installation timeout3m
sandbox.max-connectionsContainer nofile limit4096

Third-party dependencies must be declared in the generated script's PEP 723 dependencies. Host-local, legacy Docker pool, and AI Simulation executors are no longer available.

Common environment variables:

Environment VariableConfigurationPurpose
DATAAGENT_SANDBOX_DOCKER_HOSTsandbox.docker-hostPoint to a local or remote Docker Engine
DATAAGENT_SANDBOX_IMAGEsandbox.image-namePin the runtime image; use a digest in production
DATAAGENT_PYPI_INDEX_URLsandbox.package-index-urlPoint to an enterprise private PyPI proxy

Each Python task creates a separate BaseSandbox, installs dependencies and executes code in the same container, then stops and removes that container. The service-side wait timeout is the dependency installation timeout plus the code timeout plus a 30-second transport margin. requires-python is currently parsed and retained, but it does not select or validate the sandbox Python version.

See Advanced Features - Python Execution Environment Configuration for dependency syntax, security restrictions, runtime verification, and troubleshooting. See the SAA 1.1.2.2 Python Sandbox Integration Design for implementation boundaries.

6. File Storage Configuration

Configuration prefix: spring.ai.alibaba.data-agent.file

Configuration ItemDescriptionDefault Value
typeStorage type (LOCAL/OSS)LOCAL
pathLocal upload directory path./uploads
url-prefixExternal access URL prefix/uploads
image-sizeImage size limit (bytes)2097152 (2MB)
path-prefixObject storage path prefix""

7. Alibaba Cloud OSS Configuration

Configuration prefix: spring.ai.alibaba.data-agent.file.oss

Configuration ItemDescriptionDefault Value
access-key-idOSS Access Key ID-
access-key-secretOSS Access Key Secret-
endpointOSS endpoint address-
bucket-nameOSS bucket name-
custom-domainCustom domain-

8. Database Initialization

Configuration prefix: spring.sql.init

Configuration ItemDescriptionDefault ValueNotes
modeInitialization mode (always/never)neverSet to always only when initialization is explicitly required
schema-locationsTable structure script pathclasspath:sql/schema.sql
data-locationsData script pathclasspath:sql/data.sql

9. Dependency Extension

If you choose not to use Spring AI Alibaba Starter and instead manually import OpenAI or other vendor Starters:

  • Please ensure you remove the default Starter dependency to avoid conflicts.
  • You may need to manually configure ChatClient, ChatModel, and EmbeddingModel Beans.

10. Report Resources Configuration

Configuration prefix: spring.ai.alibaba.data-agent.report-template

Configuration ItemDescriptionDefault Value
marked-urlMarked.js path (Markdown rendering library)https://mirrors.sustech.edu.cn/cdnjs/ajax/libs/marked/12.0.0/marked.min.js
echarts-urlECharts path (chart library)https://mirrors.sustech.edu.cn/cdnjs/ajax/libs/echarts/5.5.0/echarts.min.js

11. Langfuse Observability Configuration

Configuration prefix: spring.ai.alibaba.data-agent.langfuse

Configuration ItemDescriptionDefault Value
enabledEnable Langfuse observabilitytrue
hostLangfuse service URL (e.g. https://cloud.langfuse.com or self-hosted)-
public-keyLangfuse project Public Key-
secret-keyLangfuse project Secret Key-

Environment variables: LANGFUSE_ENABLED, LANGFUSE_HOST, LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY

For detailed usage, refer to Advanced Features - Langfuse Observability.

Python Sandbox Verification

Unit tests that do not require Docker:

./mvnw -pl data-agent-management \
  -Dtest='PythonDependencyMetadataParserTest,PythonSandboxBootstrapBuilderTest,SandboxExecutionResultParserTest,SaaSandboxPythonCodeExecutorServiceTest,SaaSandboxRuntimeTest,PythonExecuteNodeTest,PythonWorkflowIntegrationTest' \
  test

Run the real SAA integration test while Docker is available:

docker info
./mvnw -pl data-agent-management -Dtest=SaaSandboxTaskRunnerIT test

Run the CI-equivalent checks before submission:

make format-check
make checkstyle-check
make test

A real end-to-end acceptance check must go beyond HTTP 200: the browser timeline must show dependency installation and Python execution output, a final report, and SSE event:complete. The command docker ps -a --filter name=dataagent-sandbox- must not show leftover task containers.

Learning Resources

Official Documentation

  • StateGraph Workflow Engine
  • MyBatis Data Access Framework
  • Vector Store
  • Server-Sent Events (SSE)

Contribution Guide

For detailed contribution guidelines, see CONTRIBUTING-en.md.

Contribution Types

  • Report Bugs
  • Suggest New Features
  • Improve Documentation
  • Submit Code Fixes
  • Develop New Features

Code of Conduct

  • Respect all contributors
  • Stay friendly and professional
  • Accept constructive criticism
  • Focus on project goals

Thank you for contributing to the DataAgent project!