Skip to main content

IBM Cambridge Research Center

  Technical Report: Taking Advice from Intelligent Systems: The Double-Edged Sword of Explanations

Taking Advice from Intelligent Systems: The Double-Edged Sword of Explanations

Technical Report #:11-11
Author(s): Kate Ehrlich, Susanna Kirk, John Patterson, Jamie Rasmussen, Steven Ross, Daniel Gruen

Abstract

A Collaborative User Experience Technical Report: more about CUE...


Research on intelligent systems has emphasized the benefits of providing explanations along with recommendations. But can explanations lead users to make incorrect decisions? We explored this question in a controlled experimental study with 18 professional network security analysts doing an incident classification task using a prototype cybersecurity system. The system provided three recommendations on each trial. The recommendations were displayed with explanations (called “justifications”) or without. On half the trials, one of the recommendations was correct; in the other half none of the recommendations was correct. Users were more accurate with correct recommendations. Although there was no benefit overall of explanation, we found that a segment of the analysts were more accurate with explanations when a correct choice was available but were less accurate with explanations in the absence of a correct choice. We discuss implications of these results for the design of intelligent systems.


Full Report
IUI double-edged sword pub.pdf

For more information, or to order a Technical Report, contact us.