DeepMind Open-Sources Reinforcement Learning Library TRFL

DeepMind announced today that it is open-sourcing its TRFL (pronounced ‘truffle’) library, which contains a variety of building blocks useful for developing reinforcement learning (RL) agents in TensorFlow. Created by the DeepMind Research Engineering team, TRFL is a collection of major algorithmic components that DeepMind has used internally for many of their successful agents, including DQN, DDPG and the Importance Weighted Actor Learner Architecture.

Deep reinforcement learning agents are usually composed of a large number of components which can interact in subtle ways, making it difficult for researchers to identify flaws within the large computational graphs. The DeepMind research team has introduced an approach that uses scalable distributed implementation of the v-trace agent to address this issue. The large agent codebases have made a considerable contribution in reproducing research, but lack flexibility for modification. Hence, a complementary approach is needed to provide reliable, well-tested implementation building blocks which can be used for various RL agents.

The TRFL library contains functions that can implement both advanced techniques and classical RL algorithms. TRFL also provides uncompleted algorithms which can act as complementary implementations when building a fully-functional RL agent.
As the TRFL library is still broadly used by DeepMind they will continue to maintain it and add new functionalities over time.




Posted by:   Jayne Merdith, Tendron Systems Ltd, London, UK