Challenges in Chatbot Development: A Study of Stack Overflow PostsMSR - Technical Paper
Software chatbots are flourishing these days in different domains and fields due to their benefits in saving costs, time, and effort. This is due to their capability of allowing users to communicate and control different services easily through natural language. The chatbot development requires special expertise (e.g., machine learning and conversation design) that differ from the traditional software systems. However, to date, the challenges that chatbot developers face remain fuzzy and unclear as most of the studies focused on proposing chatbots to perform particular tasks.
Therefore, in this paper, we investigate Q&A website to provide insights on the topics that chatbot developers are interested and the challenges they face. In particular, we leverage topic modeling to understand the topics that are being discussed by chatbot developers on Stack Overflow. Then, we examine the popularity and difficulty of those topics. Our results show that most of the chatbot developers are using Stack Overflow to ask about implementation guidelines. The developers discuss 12 chatbot topics (e.g., User Interface) that are grouped into five categories. Most of the posts belong to chatbot development, integration, and the natural language understanding model categories. On the other hand, we find that developers consider the posts of building and integrating chatbots topics more helpful compared to other topics. Moreover, they are facing challenges in the training of the chatbot’s model. We believe that our study guides future research to propose techniques and tools to help the community at its early stages to overcome the most popular and difficult topics that practitioners face when developing chatbots.