TensorTrade

Train RL agents to trade. Can they beat Buy-and-Hold?

TensorTrade is an open-source Python framework for building, training, and evaluating reinforcement learning agents for algorithmic trading. The framework provides composable components for environments, action schemes, reward functions, and data feeds that can be combined to create custom trading systems.

Quick Start

# Requires Python 3.12+
python3.12 -m venv tensortrade-env && source tensortrade-env/bin/activate
pip install -e .

# For training with Ray/RLlib (recommended)
pip install -r examples/requirements.txt

# Run training
python examples/training/train_simple.py

Documentation & Tutorials

📚 Tutorial Index — Start here for the complete learning curriculum.

Foundations

• The Three Pillars — RL + Trading + Data concepts
• Architecture — How components work together
• Your First Run — Run and understand output

Domain Knowledge

• Trading for RL Practitioners
• RL for Traders
• Common Failures — Critical pitfalls to avoid
• Full Introduction — New to both domains

Core Components

• Action Schemes — BSH and order execution
• Reward Schemes — Why PBR works
• Observers & Feeds — Feature engineering

Training

• First Training — Train with Ray RLlib
• Ray RLlib Deep Dive — Configuration options
• Optuna Optimization — Hyperparameter tuning

Advanced Topics

• Overfitting — Detection and prevention
• Commission Analysis — Key research findings
• Walk-Forward Validation — Proper evaluation

Additional Resources

• Experiments Log — Full research documentation
• Environment Setup — Detailed installation guide
• API Reference

---

Research Findings

We conducted extensive experiments training PPO agents on BTC/USD. Key results:

Configuration	Test P&L	vs Buy-and-Hold
Agent (0% commission)	+$239	+$594
Agent (0.1% commission)	-$650	-$295
Buy-and-Hold	-$355	—

The agent demonstrates directional prediction capability at zero commission. The primary challenge is trading frequency—commission costs currently exceed prediction profits. See EXPERIMENTS.md for methodology and detailed analysis.

---

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        TradingEnv                               │
│                                                                 │
│   Observer ──────> Agent ──────> ActionScheme ──────> Portfolio │
│   (features)      (policy)      (BSH/Orders)        (wallets)  │
│       ^                                                  │      │
│       └──────────── RewardScheme <───────────────────────┘      │
│                        (PBR)                                    │
│                                                                 │
│   DataFeed ──────> Exchange ──────> Broker ──────> Trades       │
└─────────────────────────────────────────────────────────────────┘

Component	Purpose	Default
ActionScheme	Converts agent output to orders	BSH (Buy/Sell/Hold)
RewardScheme	Computes learning signal	PBR (Position-Based Returns)
Observer	Generates observations	Windowed features
Portfolio	Manages wallets and positions	USD + BTC
Exchange	Simulates execution	Configurable commission

---

Training Scripts

Script	Description
examples/training/train_simple.py	Basic demo with wallet tracking
examples/training/train_ray_long.py	Distributed training with Ray RLlib
examples/training/train_optuna.py	Hyperparameter optimization
examples/training/train_best.py	Best configuration from experiments

---

Installation

Requirements: Python 3.11 or 3.12

# Create environment
python3.12 -m venv tensortrade-env
source tensortrade-env/bin/activate  # Windows: tensortrade-env\Scripts\activate

# Install
pip install --upgrade pip
pip install -r requirements.txt
pip install -e .

# Verify
pytest tests/tensortrade/unit -v

# Training dependencies (optional)
pip install -r examples/requirements.txt

See ENVIRONMENT_SETUP.md for platform-specific instructions and troubleshooting.

Docker

make run-notebook  # Jupyter
make run-docs      # Documentation
make run-tests     # Test suite

---

Project Structure

tensortrade/
├── tensortrade/           # Core library
│   ├── env/              # Trading environments
│   ├── feed/             # Data pipeline
│   ├── oms/              # Order management
│   └── data/             # Data fetching
├── examples/
│   ├── training/         # Training scripts
│   └── notebooks/        # Jupyter tutorials
├── docs/
│   ├── tutorials/        # Learning curriculum
│   └── EXPERIMENTS.md    # Research log
└── tests/

---

Troubleshooting

Issue	Solution
"No stream satisfies selector"	Update to v1.0.4-dev1+
Ray installation fails	Run pip install --upgrade pip first
NumPy version conflict	pip install "numpy>=1.26.4,<2.0"
TensorFlow CUDA issues	pip install tensorflow[and-cuda]>=2.15.1

---

Contributing

See CONTRIBUTING.md for guidelines.

Priority areas:

1. Trading frequency reduction (position sizing, holding periods)
2. Commission-aware reward schemes
3. Alternative action spaces

---

Community

• Discord
• GitHub Issues
• Documentation

---

License

Apache 2.0