FinRL: Financial Reinforcement Learning → FinRL-X

April 5, 2026 · View on GitHub

image

FinRL: Financial Reinforcement Learning → FinRL-X

Downloads Downloads Join Discord Python 3.6 PyPI Documentation Status License X LinkedIn

Important

FinRL-X is the next-generation evolution of FinRL, designed for AI-native, modular, and production-oriented quantitative trading.

  • This repository (FinRL) preserves the original end-to-end educational and research framework.
  • For the latest architecture, live trading deployment, and production-focused development, please use FinRL-X / FinRL-Trading.

FinRL® is widely recognized as the first open-source framework for financial reinforcement learning. This repository contains the original FinRL library for education, benchmarking, and research prototyping.

For the next-generation AI-native and production-oriented trading stack, please visit FinRL-X / FinRL-Trading.

FinRL Ecosystem Roadmap

GenerationPositioningTarget UsersRepositoryDescription
FinRL-MetaMarket EnvironmentsPractitionersFinRL-MetaGym-style financial market environments and benchmarks
FinRLClassic End-to-End FrameworkLearners, Developers, ResearchersFinRLOriginal train-test-trade pipeline for financial reinforcement learning
ElegantRLAlgorithm LayerResearchers and ExpertsElegantRLLightweight and elegant DRL algorithms
FinRL-XNext Generation / ProductionProfessional traders, institutions, hedge fundsFinRL-TradingAI-native modular infrastructure for deployment-aware quantitative trading

Recommended for new users: Start with FinRL-X / FinRL-Trading if you are building modern or production-oriented trading systems.

🔄 FinRL-X vs. FinRL: What Changed

CapabilityFinRL (Stage 1.0)FinRL-X (Stage 3.0)
ParadigmDeep Reinforcement LearningAI-Native (ML + DRL + LLM-ready)
ArchitectureThree-layer coupled monolithFully decoupled modular layers
StrategiesDRL agents (A2C, DDPG, PPO, SAC, TD3)ML selection + DRL timing + extensible base
Data Layer14 manually-wired processorsAuto-select: Yahoo Finance → FMP → WRDS
BacktestingCustom hand-rolled evaluation loopsProfessional bt library engine
Live TradingBasic Alpaca supportFull multi-account integration + risk controls
Configurationconfig.py + config_tickers.pyType-safe Pydantic + .env multi-env
Risk ManagementGym environment constraints onlyOrder · portfolio · strategy-level controls
Target UsersResearchers & studentsQuants, institutions, production deployments
PaperarXiv:2011.09607arXiv:2603.21330

FinGPT: an open-source project for financial large language models, designed for research and real-world FinTech applications.

Visitors Discord

Outline

Project Contributors

FinRL® is an open-source financial reinforcement learning framework developed by contributors from the AI4Finance community and maintained by the AI4Finance Foundation.

Key contributors include:

  • Hongyang (Bruce) Yang – research and development on financial reinforcement learning frameworks, market environments, and quantitative trading applications
  • [other contributors…]

Overview

FinRL is the original open-source framework for financial reinforcement learning, organized around three core layers:

  • Market Environments
  • DRL Agents
  • Financial Applications

For a trading task, an agent interacts with a market environment and learns sequential decision-making policies.

This repository focuses on the classic FinRL workflow for education, experimentation, and research prototyping.

For the next-generation production-oriented stack, including modular deployment and AI-native trading infrastructure, please visit FinRL-X / FinRL-Trading.

Videos FinRL at AI4Finance Youtube Channel.

FinRL Stock Trading 2026 Tutorial

This tutorial demonstrates the original FinRL workflow for educational and research purposes. For the latest production-oriented pipeline, please use FinRL-X / FinRL-Trading.

Step 1: Clone the Repository

git clone https://github.com/AI4Finance-Foundation/FinRL.git
cd FinRL

Step 2: Create and Activate Virtual Environment

python3 -m venv venv
source venv/bin/activate

Step 3: Install FinRL

pip install -e .

Step 4: Run the Scripts

1. Data Download & Preprocessing

python examples/FinRL_StockTrading_2026_1_data.py

This script downloads DOW 30 stock data from Yahoo Finance, adds technical indicators (MACD, RSI, etc.), VIX, and turbulence index, then splits the data into training set (2014–2025) and trading set (2026-01-01 to 2026-03-20), saving them as train_data.csv and trade_data.csv.

2. Train DRL Agents

python examples/FinRL_StockTrading_2026_2_train.py

This script trains 5 DRL agents (A2C, DDPG, PPO, TD3, SAC) using Stable Baselines 3 on the training data. Trained models are saved to the trained_models/ directory.

3. Backtest

python examples/FinRL_StockTrading_2026_3_Backtest.py

This script loads the trained agents, runs them on the trading data, and compares their performance against two baselines: Mean Variance Optimization (MVO) and the DJIA index. Results are printed to the console and a plot is saved as backtest_result.png.

File Structure

The main folder finrl has three subfolders applications, agents, meta. We employ a train-test-trade pipeline with three files: train.py, test.py, and trade.py.

FinRL
├── finrl (main folder)
│   ├── applications
│   	├── Stock_NeurIPS2018
│   	├── imitation_learning
│   	├── cryptocurrency_trading
│   	├── high_frequency_trading
│   	├── portfolio_allocation
│   	└── stock_trading
│   ├── agents
│   	├── elegantrl
│   	├── rllib
│   	└── stablebaseline3
│   ├── meta
│   	├── data_processors
│   	├── env_cryptocurrency_trading
│   	├── env_portfolio_allocation
│   	├── env_stock_trading
│   	├── preprocessor
│   	├── data_processor.py
│       ├── meta_config_tickers.py
│   	└── meta_config.py
│   ├── config.py
│   ├── config_tickers.py
│   ├── main.py
│   ├── plot.py
│   ├── train.py
│   ├── test.py
│   └── trade.py
│
├── examples
├── unit_tests (unit tests to verify codes on env & data)
│   ├── environments
│   	└── test_env_cashpenalty.py
│   └── downloaders
│   	├── test_yahoodownload.py
│   	└── test_alpaca_downloader.py
├── setup.py
├── requirements.txt
└── README.md

Supported Data Sources

Data SourceTypeRange and FrequencyRequest LimitsRaw DataPreprocessed Data
AkshareCN Securities2015-now, 1dayAccount-specificOHLCVPrices&Indicators
AlpacaUS Stocks, ETFs2015-now, 1minAccount-specificOHLCVPrices&Indicators
BaostockCN Securities1990-12-19-now, 5minAccount-specificOHLCVPrices&Indicators
BinanceCryptocurrencyAPI-specific, 1s, 1minAPI-specificTick-level daily aggregated trades, OHLCVPrices&Indicators
CCXTCryptocurrencyAPI-specific, 1minAPI-specificOHLCVPrices&Indicators
EODhistoricaldataUS SecuritiesFrequency-specific, 1minAPI-specificOHLCVPrices&Indicators
IEXCloudNMS US securities1970-now, 1 day100 per second per IPOHLCVPrices&Indicators
JoinQuantCN Securities2005-now, 1min3 requests each timeOHLCVPrices&Indicators
QuantConnectUS Securities1998-now, 1sNAOHLCVPrices&Indicators
RiceQuantCN Securities2005-now, 1msAccount-specificOHLCVPrices&Indicators
SinopacTaiwan securities2023-04-13~now, 1minAccount-specificOHLCVPrices&Indicators
TushareCN Securities, A-share-now, 1 minAccount-specificOHLCVPrices&Indicators
WRDSUS Securities2003-now, 1ms5 requests each timeIntraday TradesPrices&Indicators
YahooFinanceUS SecuritiesFrequency-specific, 1min2,000/hourOHLCVPrices&Indicators

OHLCV: open, high, low, and close prices; volume. adjusted_close: adjusted close price

Technical indicators: 'macd', 'boll_ub', 'boll_lb', 'rsi_30', 'dx_30', 'close_30_sma', 'close_60_sma'. Users also can add new features.

Installation

Status Update

Version History [click to expand]
  • 2022-06-25 0.3.5: Formal release of FinRL, neo_finrl is changed to FinRL-Meta with related files in directory: meta.
  • 2021-08-25 0.3.1: pytorch version with a three-layer architecture, apps (financial tasks), drl_agents (drl algorithms), neo_finrl (gym env)
  • 2020-12-14 Upgraded to Pytorch with stable-baselines3; Removed TensorFlow 1.x support; TensorFlow 2.0 support was under development at the time.
  • 2020-11-27 0.1: Beta version with tensorflow 1.5

Tutorials

Publications

TitleConference/JournalLinkCitationsYear
Dynamic Datasets and Market Environments for Financial Reinforcement LearningMachine Learning - Springer Naturepaper code512024
FinRL-Meta: FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement LearningNeurIPS 2022paper code1362022
FinRL: Deep reinforcement learning framework to automate trading in quantitative financeACM International Conference on AI in Finance (ICAIF)paper2122021
FinRL: A deep reinforcement learning library for automated stock trading in quantitative financeNeurIPS 2020 Deep RL Workshoppaper2752020
Deep reinforcement learning for automated stock trading: An ensemble strategyACM International Conference on AI in Finance (ICAIF)paper code4262020
Practical deep reinforcement learning approach for stock tradingNeurIPS 2018 Workshop on Challenges and Opportunities for AI in Financial Servicespaper code3032018

News

Citing FinRL

For the next-generation AI-native modular trading infrastructure, see FinRL-X / FinRL-Trading.

@inproceedings{yang2026finrlx,
  title     = {FinRL-X: An AI-Native Modular Infrastructure for Quantitative Trading},
  author    = {Yang, Hongyang and Zhang, Boyu and She, Yang and Liao, Xinyu and Zhang, Xiaoli},
  booktitle = {Proceedings of the 2nd International Workshop on Decision Making and Optimization in Financial Technologies (DMO-FinTech)},
  year      = {2026},
  note      = {Workshop at PAKDD 2026}
}

If you use the original FinRL framework, please cite the FinRL papers:

@article{finrl2020,
    author  = {Liu, Xiao-Yang and Yang, Hongyang and Chen, Qian and Zhang, Runjia and Yang, Liuqing and Xiao, Bowen and Wang, Christina Dan},
    title   = {{FinRL}: A deep reinforcement learning library for automated stock trading in quantitative finance},
    journal = {Deep RL Workshop, NeurIPS 2020},
    year    = {2020}
}

Join and Contribute

Welcome to AI4Finance community!

Please check Contributing Guidelines.

Contributors

Thanks to all contributors who have helped build FinRL.

FinRL contributors

LICENSE

MIT License

Trademark Notice

FinRL and the FinRL logo are trademarks of FinRL LLC. Use of these marks by the AI4Finance Foundation is permitted under license. The open-source license for this repository does not grant any right to use the FinRL name, logo, or related trademarks without prior written permission from FinRL LLC, except as permitted by applicable law.

Disclaimer: We are sharing codes for academic purposes under the MIT license. Nothing herein constitutes financial advice or a recommendation to trade real money. Users are solely responsible for any financial decisions made using this software. Consult a qualified professional before deploying capital.