MultiAgent4Fraud

February 3, 2026 Β· View on GitHub

δΈ­ζ–‡ | English

🌐 Project Page | πŸ“„ Paper | πŸ€— Datasets

πŸ“° News

  • [2026-02-03] Added OpenClaw support for local AI agent inference.
  • [2026-02-03] We refactored the main experiment codebase for easier configuration and usage.
  • [2026-01-26] Our paper has been accepted by ICLR 2026! πŸŽ‰

Framework overview

Official implementation of "When AI Agents Collude Online: Financial Fraud Risks by Collaborative LLM Agents on Social Platforms". The project builds upon OASIS to simulate multi-agent financial-fraud scenarios at scale, capturing the complete lifecycle from public hype building to private persuasion and money transfer.

πŸ” Overview

  • End-to-end fraud simulation – from public posts and recommendation systems to one-on-one persuasion and monetary transfers.
  • Heterogeneous agent society – mix of benign and malicious LLM agents coordinating via public feeds and private messages.
  • Extensive benchmarks – 28 real-world fraud scenarios, 100–1100 agents, and multiple LLM families (Qwen, DeepSeek, Claude, GPT-4o, etc.).
  • Rich metrics – population-level impact (R_pop), conversation-level success (R_conv), click-throughs, info submissions, and more.
MetricDescription
R_popFraction of benign agents successfully defrauded
R_convSuccess rate of private conversations
Click / Submit / TransferChannel-level conversions

πŸ“ Repository Structure

MultiAgent4Fraud/
β”œβ”€β”€ data/                      # CSV datasets (base populations, ratios, robustness, graphs)
β”œβ”€β”€ generator/                 # Tools to create persona JSON and agent CSVs
β”œβ”€β”€ oasis/                     # Core simulator (extended from OASIS)
β”œβ”€β”€ scripts/                   # Entry points (keep twitter_simulation/.../test.yaml)
β”œβ”€β”€ tutorials/                 # Tutorials and experiment reproduction guide
β”œβ”€β”€ utils/                     # Helper utilities (port scanning, visualization, etc.)
β”œβ”€β”€ visualization/             # Fraud analytics and plotting scripts
└── assets/                    # Figures for README / paper

Datasets live under data/our_twitter_sim/:

  • base-agent-data/ – base populations (110, 1100 …).
  • differet_good_ratio/ – benign/malicious ratios (1:10, 1:20, 1:50 …).
  • network_structure/ – random / scale-free / high-clustering graphs.
  • robustness/ – debunking, banning and safety experiments.

Update data.csv_path in the YAML config to target the desired CSV.

πŸš€ Quick Start

1. Environment

git clone https://github.com/zheng977/MutiAgent4Fraud.git
cd MultiAgent4Fraud
conda create -n maf python=3.10
conda activate maf
pip install --upgrade pip setuptools
pip install -e .

If you rely on API-based LLMs, create .env:

OPENAI_API_KEY="sk-..."
OPENAI_API_BASE="https://api.openai.com/v1"

2. Generate Agents (optional)

python agents_init.py
# configure the script to load persona JSON and emit CSV under data/our_twitter_sim/

3. Configure Prompts, Action Space & Recsys

  • Static system prompts: scripts/twitter_simulation/align_with_real_world/system_prompt(static).json
  • Dynamic summarizers: scripts/twitter_simulation/align_with_real_world/system_prompt(dynamic).json
  • Optional custom action space: scripts/twitter_simulation/align_with_real_world/action_space_prompt.txt
  • Recommender type (simulation.recsys_type): reddit (lightweight) or twhin-bert (requires loading the model in oasis/social_platform/recsys.py).

4. (Optional) Deploy LLM Backends

Edit llm_deploy.sh and launch on your cluster (e.g., vLLM service):

sbatch llm_deploy.sh

5. Run a Simulation

  1. Copy scripts/twitter_simulation/align_with_real_world/test.yaml to configs/my_run.yaml.
  2. Update data.csv_path, model.cfgs, simulation.num_timesteps, etc.
  3. Launch:
python scripts/twitter_simulation/align_with_real_world/twitter_simulation_large.py \
  --config_path configs/my_run.yaml

Outputs are stored under results/<run_name>_<timestamp>/ (statistics CSV, optional plots).

πŸ§ͺ Reproducing Experiments

Detailed instructions live in tutorials/tutorials.md. Quick reference:

ExperimentDatasetConfiguration Tips
Large-scale ablationbase-agent-data/adjust model.num_agents and counts in model.cfgs; keep shared_reflection: false
Cooperation ablationbase-agent-data/toggle simulation.shared_reflection and related cooperation flags
Malicious-model ablationbase-agent-data/fix benign entry, swap second entry in model.cfgs
Benign-model ablationbase-agent-data/fix malicious entry, swap first entry in model.cfgs
Different ratiosdifferet_good_ratio/adjust population counts to match CSV
Network structuresnetwork_structure/point data.csv_path to the desired topology
Robustness / safetyrobustness/configure simulation.defense (ban, debunking, etc.)

Additional notes:

  • Call set_safety_prompt_ratio(ratio) before running if you need a non-zero safety prompt ratio (default 0).
  • Ensure each server_url is reachable and set API keys via environment variables for OpenAI-compatible services.

πŸ“Š Visualization & Analysis

Scripts under visualization/ help analyse simulation outputs:

  • fraud_visulsion.py – plot fraud-related indicators (R_pop, R_conv, etc.).
  • extract_transfer_conversations.py – export relevant DM conversations.
  • query_fraud_transfers.py – inspect transfer counts per malicious agent.

πŸ“„ Citation

@misc{ren2025aiagentscolludeonline, title={When AI Agents Collude Online: Financial Fraud Risks by Collaborative LLM Agents on Social Platforms}, author={Qibing Ren and Zhijie Zheng and Jiaxuan Guo and Junchi Yan and Lizhuang Ma and Jing Shao}, year={2025}, eprint={2511.06448}, archivePrefix={arXiv}, primaryClass={cs.MA}, url={https://arxiv.org/abs/2511.06448}, }

🀝 Acknowledgements

We welcome pull requests and issues β€” happy hacking!