Necessity of supporting situation understanding to prevent over-trust in automation

ITOH Makoto

Department of Risk Engineering, University of Tsukuba, Tsukuba 305-8573, Japan(itoh@risk.tsukuba.ac.jp)

Abstract: It is necessary to clarify how a human operator comes to expect that an automation will perform a task successfully even beyond the limit of the automation. This paper proposes a way of modeling trust in automation by which it is possible to discuss how an operator's trust in automation becomes over-trust. On the basis of the model, an experiment using a micro world was conducted to examine whether the range of user’s expectation per se surpasses the limit of the capability of the automation. The results revealed that informing the human operator of the functional limit of capability of automation without giving an appropriate reason was effective but not perfect for avoiding human operator’s over trust. It was also shown that the understanding of the automation's limitation can be changed through experiences due to confusion about the situation, therefore, it is necessary to support adequate situation understanding of the human operator in order to prevent over-trust in automation.

Keyword: automation, trust, situation awareness, human-machine interaction