Explicit Memory

May 21, 2026 ยท View on GitHub

Authors: Taewoon Kim, Michael Cochez, Vincent Francois-Lavet, Mark Neerincx, and Piek Vossen.

Code for a deep-Q-learning agent with explicit short-term, episodic, and semantic memory that operates in RoomEnv-v1.

For the research overview, see the project page or the paper on arXiv.

This README focuses on the code, setup, training flow, and results in this repository.

Repository layout

  • agent/: agent implementation and memory-related components
  • train.py: runs training in RoomEnv-v1
  • train.yaml: configuration for training runs
  • models/: saved models and related outputs
  • figures/: generated plots and analysis figures
  • paper/: paper source and paper figures
  • test/: tests

Prerequisites

  1. Python 3.10 or higher
  2. A virtual environment is recommended
  3. Install the requirements with pip install -r requirements.txt

Run training

python train.py

Configure the training run in train.yaml. Training outputs, checkpoints, and derived figures are written into the repository outputs used by the analysis scripts.

Training setup

The project trains a DQN-based agent that decides what to do with the oldest short-term memory when the short-term buffer is full.

  • forget it
  • move it to episodic memory
  • move it to semantic memory

The repository includes both a semantic-scratch variant and a semantic-pretrained variant that starts with ConceptNet-based world knowledge.

Results

Average loss, training.Average total rewards per episode, validation.Average total rewards per episode, test.
Average total rewards, varying capacities, test.

Also check models/ for saved training runs and paper/ for the paper source and additional figure assets.

Further reading

Cite our paper

@article{Kim_Cochez_Francois-Lavet_Neerincx_Vossen_2023,
  title={A Machine with Short-Term, Episodic, and Semantic Memory Systems},
  volume={37},
  url={https://ojs.aaai.org/index.php/AAAI/article/view/25075},
  DOI={10.1609/aaai.v37i1.25075},
  number={1},
  journal={Proceedings of the AAAI Conference on Artificial Intelligence},
  author={Kim, Taewoon and Cochez, Michael and Francois-Lavet, Vincent and Neerincx, Mark and Vossen, Piek},
  year={2023},
  month={Jun.},
  pages={48-56}
}