Response Quality in Human-chatbot Collaborative Systems

Simple item page
dc.contributor.authorAhuja, Namanen
dc.contributor.committeechairJiang, Jiepuen
dc.contributor.committeememberRibbens, Calvin J.en
dc.contributor.committeememberKarpatne, Anujen
dc.contributor.departmentComputer Scienceen
dc.date.accessioned2021-11-19T07:00:06Zen
dc.date.available2021-11-19T07:00:06Zen
dc.date.issued2020-05-27en
dc.description.abstractWe study human-chatbot collaborative conversation systems that enable humans to leverage AI chatbot outputs during an online conversation with others. We evaluate response quality in two collaborative systems and compare them with human-only and chatbot-only settings. Both collaborative systems present AI chatbot results as suggestions but encourage the synthesis of human and chatbot responses to different extents. We also examine the influence of chatbot choices, including both retrieval-based and generation-based methods, and the number of suggestions on collaborative systems. Experimental results show that our collaborative systems can significantly improve the efficiency to formulate a response and improve its quality compared with a human-only system while sacrificing the fluency and humanness of the messages. Compared with a chatbot, collaborative systems can provide answers that are more fluent, human-like, and informative. We also found that the retrieval-based chatbots perform better than the generation-based one from all aspects. The optimal number of chatbot suggestions is one, and showing more suggestions has reduced user efficiency.en
dc.description.abstractgeneralArtificial Intelligence (AI) systems have become remarkably interactive and accurate with them becoming an integral part of our life. The increasing use of personal assistants like Siri and the application of AI in important real-world tasks such as medical imaging and diagnosis show that AI can perform as good as trained human experts. Organizations today are expanding at a rapid rate and need to service millions of customers concurrently to remain competitive in the market. With the recent success of AI chatbots, the collaboration of Human and AI to augment customer service management is one of the most sought out solutions to this requirement. A service flow where virtual agents and people work together can be a boon to the industry by making the human agents smarter with a bot "whispering" in their ears. We present the design of various collaborative systems we have developed and discuss the improvements in response efficiency and quality due to them in multiple online user experiments. The results of this study can be used to improve conversational chat systems that assist human agents to improve their response time and quality and identify features of the AI agent that are most beneficial for improving the conversation.en
dc.description.degreeMaster of Scienceen
dc.format.mediumETDen
dc.identifier.othervt_gsexam:26129en
dc.identifier.urihttp://hdl.handle.net/10919/106683en
dc.publisherVirginia Techen
dc.rightsIn Copyrighten
dc.rights.urihttp://rightsstatements.org/vocab/InC/1.0/en
dc.subjectConversational systemsen
dc.subjectchatboten
dc.subjecthuman-AI collaborationen
dc.subjectdialogue systemsen
dc.titleResponse Quality in Human-chatbot Collaborative Systemsen
dc.typeThesisen
thesis.degree.disciplineComputer Science and Applicationsen
thesis.degree.grantorVirginia Polytechnic Institute and State Universityen
thesis.degree.levelmastersen
thesis.degree.nameMaster of Scienceen

Files

Original bundle
Now showing 1 - 3 of 3
Thumbnail Image
Name:
Ahuja_N_T_2020.pdf
Size:
3.89 MB
Format:
Adobe Portable Document Format
Download
Name:
Ahuja_N_T_2020_support_4.docx
Size:
109.03 KB
Format:
Microsoft Word XML
Description:
Supporting documents
Download
Thumbnail Image
Name:
Ahuja_N_T_2020_support_3.pdf
Size:
98.52 KB
Format:
Adobe Portable Document Format
Description:
Supporting documents
Download

Collections

Masters Theses