Anomaly Detection with Visual Question Answering
Report Number:
ARL-TR-9817
October 18, 2023
Approved for public release: distribution is unlimited.
Author(s):
Stephanie M Lukin and Rahul Sharma
Abstract:Anomaly detection is critical for many different use-cases, such as identifying safety hazards to potentially prevent disasters. Developing the capability for a human-robot team to ask targeted questions would be critical to quickly identify a violation of protocol and then quickly take action to rectify the situation. In this report, we experiment with how visual question answering algorithms can be used with a set of carefully constructed questions to detect anomalies in a virtual makerspace and a real-world alleyway. Our exploratory results show improvement over a random baseline and we discuss challenges for future work.
