SAC

November 25, 2020 · View on GitHub

Implementation of Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

Added another branch for Soft Actor-Critic Algorithms and Applications -> SAC_V1.

Soft Q-Learning uses the following objective function instead of the conventional expected cumulative return:

$J( \pi ) = \sum_{t=0}^T E_{( s_{t}, a_{t}) \sim{ \rho _{t} }} [r(s_{t}, a_{t}) + \alpha H(.|s_{t})]$

The entropy term is also maximized which have two major benefits:

The exploration will be intelligently tuned and maximized as much as need, so the exploration/exploitation trade off is well satisfied.
It prevent the learning procedure to get stuck in a local optima which results to a suboptimal policy.

Demos

Humanoid-v2	Walker2d-v2	Hopper-v2

Humanoid-v2	Walker2d-v2	Hopper-v2

pip3 install -r requirements.txt

python3 main.py

You may use Train flag to specify whether to train your agent when it is True or test it when the flag is False.
There are some pre-trained weights in pre-trained models dir, you can test the agent by using them; put them on the root folder of the project and turn Train flag to False.

All credits goes to @pranz24 for his brilliant Pytorch implementation of SAC.
Special thanks to @p-christ for SAC.py