HarDRL: Acceleration of Deep Reinforcement Learning Algorithms in Hardware

Author(s):

Vinod K. Mishra and Kanad Basu

Abstract:

The proliferation of Deep Reinforcement Learning (DRL) has led to its adoption in solving several real-world problems (e.g., medical diagnosis, drug discovery, and self-driving vehicles). Furthermore, in recent years, DRL algorithms have been incorporated into military and defense applications such as remote surgical and evacuation support, cybersecurity, and target recognition. However, because conventional DRL implementations use general-purpose CPUs and graphical processing units, they are resource intensive. They incur significant power, area, and latency overheads, leading to inefficiency and inefficacy. As a solution to these problems, we propose hardware (HW) acceleration of DRL algorithms for real-time deployment. This involves the 8-bit quantization of model parameters and inputs, followed by novel dictionary-based compression and HW-based decompression strategies, resulting in more than four-times reduction in memory footprint. This leads to a maximum memory savings of 80.73% while incurring a negligible area overhead of 0.022% and power overhead of 0.002%, thereby facilitating the efficient deployment of DRL algorithms in HW.