Generalization between Two Human–Automation Collaborative Tasks Via Data-based Transfer Learning

Author(s):

Torin Adamson, Liz DiGioia, Yazied Hansan, Ava Naffah, Matthias Mehl, Lydia Tapia, and Evan C. Carter

Abstract:

When collaborating with AI, human behavior can evolve unpredictably outside a controlled lab environment. Improving our understanding of these variations in behavior requires a larger volume of data. Our approach gamifies human–AI collaboration with a mobile video game, making it feasible to collect sufficient data by repeating experiments daily over a long period of time. This data can then be analyzed to generalize knowledge of human–AI collaboration across different tasks. To demonstrate this, we presented 83 participants with one of two dynamic obstacle-avoidance tasks over the course of a 180-day study. The two tasks differed in the role played by the human. They consisted of a supervisory role (where the human takes control from the AI when they believe it is necessary) and a defense role (where the human prevents objects from reaching the AI-controlled payload). To evaluate whether collaboration behavior could be generalized, a transfer learning model was trained on participant data from the supervisor task to predict behavior in the defensive task. The capability to generalize using transfer learning was demonstrated when models showed improved prediction of the defensive task after data collected from both experimental tasks was combined in training sets in unequal measure.