Application of Reinforcement Learning to the Problem of Multiple UAVs Attacking a Counter-UAV Turret

Author(s):

James Humann, Michael Dorothy, Stephen Nogar, and Christopher Kroninger

Abstract:

This report explores the use of reinforcement learning (RL) in the development of collaborative strategies for multiple unmanned aerial vehicles (UAVs) to attack a counter-UAV turret. We first validate the use of RL by recreating the optimal one- and two-UAV strategies and comparing the RL strategies to the theoretical best strategies and a selection of naïve strategies. We then discuss how the simplifications used in modeling and training could be applied to higher numbers of UAVs. In the case of the 1 vs. 1 game, tabular Q-learning was used to train an agent that could win 95.6% of the games that an optimal agent could win. This was 18.6% more than a naïve approach that guided an agent straight at the turret. In the more complex 2 vs. 1 case, a soft actor-critic algorithm was trained that could win 99.4% of the games that an optimal agent won. The success of the RL-trained agents serves as a proof of concept for applying RL to this domain. It is a promising direction for future work in deriving successful strategies for 3 vs. 1 and higher games. The RL algorithms, parameters, and modeling approach reported here should transfer to speed training in this problem space.