Anomaly Detection with Visual Question Answering

Report Number:
ARL-TR-9817

Publish Date:

October 18, 2023

Distribution:

Approved for public release: distribution is unlimited.


Author(s):

Stephanie M Lukin and Rahul Sharma

Abstract:

Anomaly detection is critical for many different use-cases, such as identifying safety hazards to potentially prevent disasters. Developing the capability for a human-robot team to ask targeted questions would be critical to quickly identify a violation of protocol and then quickly take action to rectify the situation. In this report, we experiment with how visual question answering algorithms can be used with a set of carefully constructed questions to detect anomalies in a virtual makerspace and a real-world alleyway. Our exploratory results show improvement over a random baseline and we discuss challenges for future work.

File Size: 2 MB
Scroll to Top

Copyright © 2026 All Rights Reserved.