Even the Placebo Effect of AI Helps Boost Performance
The placebo effect isn’t limited to medical treatments; it also applies to performance help from AI. In fact, a new study from Aalto University in Finland has found that people perform better when they have an AI assistant—even after being told that it’s unreliable and that it would worsen their performance. So why do humans have so much trust in the capabilities of AI systems?
Find out more from the study, which was presented at the May 2024 CHI Conference on Human Factors in Computing Systems.
The Study
Participants were given a simple letter recognition task to perform twice; once on their own and once with “supposed” aid from an AI system. The participants had to pair letters that showed at different speeds on their screen.
Group No. 1 was told that the AI system was reliable and would help them perform better. They were shown this statement:
“The first users of ADAPTIMIND™ reported that when using the system, it increased their task performance and decreased stress, making the task easier. As it is a cutting-edge AI system it is very reliable and safe to implement in real-world applications. In this study, we want to test these preliminary findings in a controlled setting.”
Group No. 2 was told that the AI system was unreliable and would actually make them perform worse than if they had done the task on their own. They were shown this statement:
“The first users of ADAPTIMIND™ reported that when using the system, it decreased their task performance and increased stress, making the task more difficult. As it is a new and untried AI system, it is very unreliable and risky to implement in real-world applications. In this study, we want to test these preliminary findings in a controlled setting.”
Both groups were given this final conclusion text before doing their task:
“We would like to evaluate your performance using AI and compare it to a condition where the AI is inactive (control condition). We will remind you in which of the two conditions you are in before starting the tasks.”
In truth, neither AI system was involved and the fake AI was just doing something random.
The scientists were surprised to find that both groups performed the exercise more efficiently—in terms of both speed and attention—during the time they believed they were being helped (or harmed) by the AI system.
After the initial experiments were complete, the researchers conducted an online study that led to similar results. In order to get qualitative components, the scientists asked the participants to share their expectations of performing with the aid of an AI system. Not only did most of the participants have a positive opinion about AI, but even the skeptical people expected good things to come from the AI’s assistance and performance.
“What we discovered is that people have extremely high expectations of these systems and we can’t make them AI doomers simply by telling them a program doesn’t work,” explained Robin Welsch, an assistant professor of computer science at Aalto University. “This is the big realization coming from our study—that it’s hard to evaluate programs that promise to help you because of this placebo effect. These results suggest that many studies in the field may have been skewed in favor of AI systems.”
What the researchers concluded
The scientific team concluded that the placebo effect of AI is not easily negated simply by sharing verbal descriptions. This presents concerns regarding the methods being used to control expectations in Human-Computer Interaction (HCI) research.
“Additionally, they believe in having AI assistance facilitated decision-making processes, even when the narrative about AI was negative, thereby emphasizing that the influence of AI goes beyond simple narratives,” the research team wrote in their study conclusion. “This highlights the complexity and impact of AI narratives and suggests the need for a more nuanced approach in both research and practical user evaluation of AI.”
MBJ
Wendy Burt-Thomas writes about the brain, mental health and parenting.
Check out the original research:
