Human–Machine Integration (HMI) Approach for Small Unit Single Target Tracking in Urban Environment

Author(s):

Andrew Tweedell, Paul Sabbagh, Sangjin Han, Joseph Conroy, Jinho Kim, Osben Toulson, Richard Diego, William Maslin, and Jonathan Touryan

Abstract:

Near future US Army Infantry squads will soon deploy with increasingly intelligent technologies including small arms, unmanned vehicles (UAVs and UGVs), integrated augmented-reality (AR) heads-up displays (HUDs), and other tactical sensors to provide small-unit lethality overmatch on the ground. Currently, these system or platform concepts are not well integrated with each other or with the squad. This effort seeks to establish technologies and methodologies for investigating how these systems should interact with each other and with Soldiers in operational settings. The purpose of this report is to share results demonstrating real-time, multisystem (HUD, weapon, UAS), cross-platform integrated data capturing capabilities during military operations on urbanized terrain, featuring a single target acquisition and handoff exercise between squad members. Two sensor-ed rifle systems, two HoloLens 2 (HL2) AR wearable glasses, and one BlueUAS Teal UAS were simultaneously used by a small team (two riflemen, one UAS operator, and one squad leader [SL]) to identify and track a single moving human target as the target navigated a complex environment. The UAS and rifle systems successfully identified and tracked the target while providing GPS coordinates across the data network. This localization data were successfully ingested by the HL2 systems and used to 1) display horizontal and aerial video to the SL to determine the target threat level, and 2) display target location and threat level to the riflemen relative to their spatial location. The culminating proof-of-concept demonstration successfully demonstrated the need for a Soldier-centric integration plan for deploying small-unit lethality technologies and provides a technological pathway for future human–machine integration research for dismounted operations.