This recent master research project has tackled the question by observing and interviewing users in a large organization. Information Sciences student Sachira Tintar has tried to improve the responsivity of a chatbot, which is currently being implemented as a replacement for the usual helpdesk in the onboarding process of new staff.
The first question is, evidently, if the chatbot – which is not equipped with a large language model such as ChatGPT – can understand the user’s question and give a correct answer. This can be tested and improved by doing a good-old, agile, use-case analysis, through a user-centered requirements engineering process.
The second question that comes up in this research is more complex and needs more exploration and reflection: how do people evaluate the interaction with the chatbot? The interviews show how users tend to humanize the chatbot, by expecting it – not only to understand a complex question in natural language, but also – to behave human-like, respectful, understanding, polite, even whitty or messy as a person. Why do people appreciate these properties in communication, when they know they are not talking to a human? Some users express experiencing an uncanny feeling when the chatbot seems to become too personal by exhibiting human-like behavior.
When I have a problem, and I want understanding, I rather talk to a person. Could this person be a bot? That depends how I — myself, a human — perceive the chatbot. Do I expect it to feel empathy and understanding when I pose my question? An why does it bring up in me so many emotions, uncanny feelings, anger, confusion, when I am interacting with a chatbot?
This study brings up many interesting questions for reflection. It shows how digital society & global citizenship projects are more than just about learning digital skills. If the goal of the DEP program is to contribute to 20 million ICT professionals in Europe by 2030, we are in need of the social, the ethical, the humanist perspective.