Using Large Language Models (LLMs) to Summarize Mission Data for Mission Planning and After Action Review (AAR)

Author(s):

Benjamin T. Files and Kimberly A. Pollard

Abstract:

U.S. Army systems use sensors and computer algorithms to generate and capture large amounts of complex data. These data could support generation of reports, graphics, and summaries for after action reviews, team-to-team communication, and other purposes. Here, we describe a conceptual solution that would use automated mission summaries to convert relevant data into natural language text narratives. We propose a hierarchical approach that converts low-level events and observations into sentences and leverages the ability of large language models to effectively summarize text to produce narratives at an appropriate level. In pursuit of this goal, we explored candidate methods for generating first-level sentences from raw mission data gathered from an interactive simulator exercise. We focus on two key classes of mission data: firing events and vehicle formations. Results with firing events demonstrate the general principle that a language model can convert event data into natural language sentences with an accuracy that matches a human, although the nature of its errors differed from that of a human. Spatial reasoning to recognize formations is not a strength of language models. We arrived at a system that could reliably recognize categories of formations, but we caution that the system was neither flexible nor likely to generalize.