Diversity Is All You Need (DIAYN) Implementation using RLkit

Implementation of Proximal Diversity Is All You Need (DIAYN) using the reinforcement learning framework RLKit by vitchyr. Installation for RLKit is specified in the original README for RLKit.

• Diversity Is All You Need (DIAYN)

  • Example script
  • Paper

• Proximal Policy Optimization (PPO)

  • Example script
  • Paper
  • Other References
    • SpinningUp documentation
    • Lilian Weng's blog

Running the Example Script for DIAYN

First, run the following command for training the sub-policies:

python examples/diayn.py <NAME_OF_ENVIRONMENT>

In addition, you can specify the number of skills that DIAYN is going to learn. The default is set at 10.

python examples/diayn.py <NAME_OF_ENVIRONMENT> --skill_dim <NUMBER_OF_SKILLS>

After training DIAYN, a file is saved onto data/DIAYN_<NUMBER_OF_SKILLS>_<ENVIRONMENT>_<DATE_AND_TIME>. Use the saved file to train the manager using PPO.

python examples/ppo_diayn.py <NAME_OF_ENVIRONMENT> <PATH_TO_SUB_POLICY>/params.pkl

Run the following command for visualizing the trained policies:

python scripts/run_policy_diayn.py <PATH_TO_SUB_POLICY>/params.pkl

Here is an example implementation result on the OpenAI Gym environment, Bipedal Walker-v2:

Intrinsic Reward Learning Curve:

Policy Loss Learning Curve:

Running the Example Script for PPO

Run the following command:

python examples/ppo.py

Here is an example implementation result on the OpenAI Gym environment, Bipedal Walker-v2:

Other Major Contributors

• seann999